Generative modeling represents a process or probability distribution that assigns likelihoods to observations or produces new observations. A fitted model can therefore evaluate how typical a record is and, in suitable settings, generate synthetic data. section 11.2.3 section 5.6
A Gaussian mixture model (GMM) treats data as arising from multiple weighted multivariate Gaussian clusters. Fitting adjusts cluster parameters to maximize the likelihood of the training data; afterward, unlikely records can be treated as outliers. section 5.6
Generative models support several related uses:
- Generate realistic synthetic records by selecting a cluster according to its weight and sampling from that cluster's Gaussian distribution. section 11.2.3
- Create more unusual test data by modifying cluster means or covariance matrices, though ordinary sampling mostly produces typical inliers. section 11.2.3
- Model images with variational autoencoders (VAEs) or GANs; in outlier detection, VAEs usually rely on reconstruction error rather than generation itself. section 16.3.1 section 16.2.2
Interpretation depends on assumptions. GMMs fit best when several clusters are reasonably Gaussian, need sufficiently large datasets, and can overfit covariance parameters. High-dimensional or strongly non-Gaussian data may require restricted covariance forms for faster, more stable fitting, at the cost of approximation. section 5.6
Baseball data
A baseball dataset is cleaned, one-hot encoded, and partly trimmed with Isolation Forest before selecting a GMM configuration by Bayesian information criterion. Samples from the fitted model preserve associations between features and generally look more typical than some real records; expanding covariance matrices creates a broader synthetic range containing many likely outliers. section 11.2.3
