Expectation maximization
Expectation maximization is an iterative algorithm for estimating model parameters with latent or unobserved variables by alternating latent-variable estimation with parameter optimization.
Machine Learning Algorithms in Depth
The expectation-maximization algorithm iteratively estimates latent variables and maximizes expected complete-data likelihood to fit models with unobserved variables.
- How it is used
- Fits Gaussian mixture models without observed component assignments by alternating soft assignment and parameter estimation.
- How deep it goes
- Derives responsibilities and ML updates for mixture proportions, means, and covariances, then provides GMM pseudocode and Python implementation.
- Examples
- Uses generated Gaussian-mixture data, Python implementation, clustering results, and a log-likelihood plot.
- What you can do after
- Build it from scratch for a GMM by computing responsibilities and updating mixture proportions, means, and covariances.
Read how Machine Learning Algorithms in Depth explains expectation maximization →
Data Without Labels
The expectation-maximization algorithm is an iterative statistical method that alternates estimating latent-variable assignments and optimizing model parameters.
- How it is used
- Estimates Gaussian mixture parameters and latent cluster memberships when assignments are uncertain or data are incomplete.
- How deep it goes
- Explains the E-step, parameter updates, and repetition until a good fit, while leaving detailed mathematical derivation optional.
- Examples
- Uses Gaussian mixture clustering with probabilistic assignments across overlapping components.
- What you can do after
- Apply EM to estimate Gaussian mixture parameters and probabilistic memberships for overlapping datasets.
Read how Data Without Labels explains expectation maximization →
Math and Architectures of Deep Learning
An iterative algorithm for fitting latent-variable models by alternating between computing posterior responsibilities and updating parameters.
- How it is used
- Uses alternating posterior estimation and parameter updates to perform maximum likelihood fitting for Gaussian mixture models.
- How deep it goes
- Explains initialization, E-step responsibilities, M-step parameter formulas, likelihood stopping, and PyTorch computation.
- Examples
- Uses an unlabeled point cloud, iterative sampled clouds, weighted parameter formulas, and a PyTorch GMM implementation.
- What you can do after
- Apply it via PyTorch to fit Gaussian mixture parameters from unlabeled training data.
Read how Math and Architectures of Deep Learning explains expectation maximization →
Outlier Detection in Python
Expectation maximization is an iterative algorithm for estimating parameters in models with latent or incomplete assignments by alternating parameter evaluation and adjustment.
- How it is used
- Uses iterative likelihood-based fitting to estimate Gaussian mixture parameters for clustered-data modeling and outlier detection.
- How deep it goes
- Explains initialization, probability evaluation, parameter adjustment, and stopping criteria, but does not derive EM mathematically.
- Examples
- Uses synthetic two-dimensional data containing three clusters and separately generated outliers.
- What you can do after
- Apply it via GaussianMixture to fit mixture models and identify records with unusually low generated-data probability.
Read how Outlier Detection in Python explains expectation maximization →
Related concepts
- Gaussian mixture modeluses · A Gaussian mixture model is a probabilistic model representing a distribution as a weighted combination of multiple Gaussian component distributions.
- expectation stephas part · The expectation step computes expected values or posterior distributions of unobserved variables conditional on current model parameters.
- maximization stephas part · The maximization step is the phase of expectation-maximization that updates model parameters by optimizing the expected complete-data objective given current latent-variable estimates.
- likelihooduses · A likelihood is the probability of observed data under a specified probabilistic model, regarded as a function of candidate hypotheses or parameter values.
- hidden Markov modeluses · A hidden Markov model is a probabilistic sequence model in which latent states form a Markov process and generate observable data according to state-dependent probabilities.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.