Expectation maximization (EM) fits Gaussian mixture model parameters when component assignments are unobserved. It alternates between estimating assignment probabilities and maximizing parameters under those soft assignments. section 6.9.4
Each iteration separates into two steps:
- E-step: compute each data point’s posterior responsibility for every mixture component using the current parameters. section 6.9.4
- M-step: update mixture weights, means, and covariances using responsibility-weighted counts, centroids, and covariances. section 6.9.4
- Repeat until the likelihood stops increasing, beginning from randomly initialized mixture parameters. section 6.9.4
Fitting a Gaussian mixture
Given an unlabeled point cloud, EM begins with provisional components, calculates log responsibilities with component log probabilities and mixture weights, then updates π, μ, and Σ. After successive iterations, sampled points from the fitted mixture almost match the target cloud. section 6.9.4
Figure 6.12 Progression of maximum likelihood estimation for GMM parameters
|
(a) Training data point cloud (target for fitting) |
|
(b) Fitted GMM’s sample point cloud at step 0 |
|
(c) Fitted GMM’s sample point cloud at step 5 |
|
(d) Fitted GMM’s sample point cloud at step 15. It almost matches the target. |
Trace the fitted Gaussian mixture from initialization through later iterations as its sampled point cloud approaches the training data. section 6.9.4
