Expectation maximization (EM) estimates model parameters when latent variables are unknown, data may be incomplete, or observations can arise from multiple Gaussian distributions. section 5.6.1
In Gaussian mixture modeling, each cluster is represented by a Gaussian distribution with a mean, covariance or variance, and weight. EM alternates between estimating probabilistic cluster assignments and updating these parameters. section 5.6 section 5.6.1
The iterative procedure follows two linked steps:
- Expectation step: calculate the probability that each data point was generated by each Gaussian, producing soft cluster assignments. section 5.6.1
- Maximization step: use those latent assignments to estimate updated model parameters, including distribution means, covariances or variances, and weights. section 5.6.1 section 5.6
- Repeat both steps until the latent assignments and parameters form a sufficiently good fit to the data. section 5.6.1
Overlapping Gaussian clusters
For a dataset of n items and k Gaussian components, EM assigns every item probabilities of belonging to the components rather than forcing a single centroid-based assignment. This makes it suitable for overlapping data modeled through GMM. section 5.6.1 section 5.9
