Expectation maximization
Expectation maximization is an iterative algorithm for estimating model parameters with latent or unobserved variables by alternating latent-variable estimation with parameter optimization.
Machine Learning Algorithms in Depth
The expectation-maximization algorithm iteratively estimates latent variables and maximizes expected complete-data likelihood to fit models with unobserved variables.
- How it is used
- Fits Gaussian mixture models without observed component assignments by alternating soft assignment and parameter estimation.
- How deep it goes
- Derives responsibilities and ML updates for mixture proportions, means, and covariances, then provides GMM pseudocode and Python implementation.
- Examples
- Uses generated Gaussian-mixture data, Python implementation, clustering results, and a log-likelihood plot.
- What you can do after
- Build it from scratch for a GMM by computing responsibilities and updating mixture proportions, means, and covariances.
Read how Machine Learning Algorithms in Depth explains expectation maximization →
Data Without Labels
The expectation-maximization algorithm is an iterative statistical method that alternates estimating latent-variable assignments and optimizing model parameters.
- How it is used
- Estimates Gaussian mixture parameters and latent cluster memberships when assignments are uncertain or data are incomplete.
- How deep it goes
- Explains the E-step, parameter updates, and repetition until a good fit, while leaving detailed mathematical derivation optional.
- Examples
- Uses Gaussian mixture clustering with probabilistic assignments across overlapping components.
- What you can do after
- Apply EM to estimate Gaussian mixture parameters and probabilistic memberships for overlapping datasets.
Read how Data Without Labels explains expectation maximization →
Math and Architectures of Deep Learning
An iterative algorithm for fitting latent-variable models by alternating between computing posterior responsibilities and updating parameters.
- How it is used
- Uses alternating posterior estimation and parameter updates to perform maximum likelihood fitting for Gaussian mixture models.
- How deep it goes
- Explains initialization, E-step responsibilities, M-step parameter formulas, likelihood stopping, and PyTorch computation.
- Examples
- Uses an unlabeled point cloud, iterative sampled clouds, weighted parameter formulas, and a PyTorch GMM implementation.
- What you can do after
- Apply it via PyTorch to fit Gaussian mixture parameters from unlabeled training data.
Read how Math and Architectures of Deep Learning explains expectation maximization →
Outlier Detection in Python
Expectation maximization is an iterative algorithm for estimating parameters in models with latent or incomplete assignments by alternating parameter evaluation and adjustment.
- How it is used
- Uses iterative likelihood-based fitting to estimate Gaussian mixture parameters for clustered-data modeling and outlier detection.
- How deep it goes
- Explains initialization, probability evaluation, parameter adjustment, and stopping criteria, but does not derive EM mathematically.
- Examples
- Uses synthetic two-dimensional data containing three clusters and separately generated outliers.
- What you can do after
- Apply it via GaussianMixture to fit mixture models and identify records with unusually low generated-data probability.
Read how Outlier Detection in Python explains expectation maximization →
What to know first
- K-means clusteringK-means clustering is an algorithm that partitions data into a specified number of clusters by iteratively assigning observations to the nearest centroid and updating each centroid to the mean of its assigned observations.
- maximum likelihood estimationMaximum likelihood estimation is a statistical method that selects parameter values maximizing the likelihood of observed data under a specified probabilistic model.
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
- varianceVariance is the expected squared deviation of a random variable from its mean and measures the dispersion of its values.
Related concepts
- Gaussian mixture modeluses · A Gaussian mixture model is a probabilistic model that represents a distribution as a weighted combination of Gaussian component distributions.
- expectation stephas part · The expectation step computes expected values or posterior distributions of unobserved variables conditional on current model parameters.
- maximization stephas part · The maximization step is the phase of expectation-maximization that updates model parameters by optimizing the expected complete-data objective given current latent-variable estimates.
- hidden Markov modeluses · A hidden Markov model is a probabilistic sequence model in which latent states form a Markov process and generate observable data according to state-dependent probabilities.
- likelihooduses · A likelihood is the probability of observed data under a model, viewed as a function of the model’s parameters or candidate hypotheses.
Questions about expectation maximization
What is expectation maximization?
Expectation maximization is an iterative algorithm for estimating model parameters with latent or unobserved variables by alternating latent-variable estimation with parameter optimization.
Which Manning books explain expectation maximization?
Machine Learning Algorithms in Depth, Data Without Labels, Math and Architectures of Deep Learning and Outlier Detection in Python.
What should I know before learning expectation maximization?
Manning books that teach expectation maximization usually introduce K-means clustering, maximum likelihood estimation, mean and variance first.
What are the parts of expectation maximization?
Expectation maximization includes expectation step and maximization step.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



