Maximum likelihood estimation
Maximum likelihood estimation is a statistical method that selects parameter values maximizing the likelihood of observed data under a specified probabilistic model.
Timeless Algorithms, The Foundational Ideas
A maximum likelihood estimate is a parameter value that maximizes a statistical model’s likelihood function.
- How it is used
- Uses likelihood maximization as a unifying framework for estimating parameters across statistical and machine-learning models.
- How deep it goes
- Derives the normal-mean estimate and regression likelihoods, while deferring detailed neural-network treatment.
- Examples
- Works through coin tosses, a normal sample mean, and regression on study hours and exam scores.
- What you can do after
- Evaluate/understand estimates by identifying their likelihood, model assumptions, and limitations.
Read how Timeless Algorithms, The Foundational Ideas explains maximum likelihood estimation →
Math and Architectures of Deep Learning
A parameter-estimation method that chooses values maximizing the likelihood of the observed data under a probabilistic model.
- How it is used
- Fits probability-model parameters from observed training data, especially for unsupervised distributions and supervised optimization.
- How deep it goes
- Derives Gaussian closed-form estimates, presents negative-log-likelihood optimization, and distinguishes iterative Gaussian-mixture fitting.
- Examples
- Uses adult-height samples to fit a Gaussian and PyTorch code to calculate a height-range probability.
- What you can do after
- Apply it via PyTorch to estimate Gaussian parameters from samples and evaluate probabilities of value ranges.
Read how Math and Architectures of Deep Learning explains maximum likelihood estimation →
Statistics Slam Dunk
Maximum likelihood estimation is a parameter-estimation method that selects values maximizing the likelihood of the observed data under a model.
- How it is used
- Uses likelihood-based logistic regression to classify playoff qualification without producing impossible probabilities.
- How deep it goes
- Defines the logit equation and demonstrates glm() fitting, then defers broader model-result evaluation.
- Examples
- Demonstrates playoff prediction from z_o_pts and compares it with a model using z_pts.
- What you can do after
- Apply it via base R glm() to fit binary models and evaluate thresholded playoff predictions.
Read how Statistics Slam Dunk explains maximum likelihood estimation →
Causal AI
Maximum likelihood estimation selects parameter values that maximize the probability or likelihood of the observed data.
- How it is used
- Estimates probability-distribution parameters from observed data, including conditional probability tables in a causal graphical model.
- How deep it goes
- Defines the objective, explains log-likelihood and optimization, and works through categorical count proportions without deriving a general optimizer.
- Examples
- Uses a conditional probability estimated from categorical counts and the default estimator in pgmpy’s BayesianNetwork.fit.
- What you can do after
- Apply it via pgmpy’s BayesianNetwork.fit to estimate categorical conditional probability tables, or recognize when numerical optimization is needed.
Grokking Statistics
A parameter-estimation technique that selects parameter values maximizing the likelihood of the observed data.
- How it is used
- Fits logistic-regression coefficients by maximizing the likelihood of observed binary outcomes.
- How deep it goes
- Defines likelihood fitting and contrasts it with least squares, but defers the mathematical derivation to libraries.
- Examples
- Uses hours without a nap, toddler meltdown outcomes, a fitted curve, and a predicted probability at 4.78 hours.
- What you can do after
- Apply it via scikit-learn to fit logistic-regression parameters and predict probabilities from observed data.
Read how Grokking Statistics explains maximum likelihood estimation →
Also covered in
What to know first
- Bayesian inferenceBayesian inference updates probability distributions over hypotheses or unknown quantities by combining prior information with observed evidence and a likelihood model.
- independent variableAn independent variable is an input or explanatory variable used to model, explain, or predict variation in an outcome, without necessarily causing it.
- gradient descentGradient descent is an iterative optimization method that updates parameters in the direction opposite an objective function’s gradient to reduce the objective’s value.
- prior probabilityA prior probability or distribution is a probability or probability distribution assigned to an event, hypothesis, or unknown quantity before relevant evidence is incorporated.
Related concepts
- logistic regressionused by · Logistic regression is a statistical classification model that estimates class probabilities by applying a logistic or softmax function to a linear combination of input features.
- likelihooduses · A likelihood is the probability of observed data under a model, viewed as a function of the model’s parameters or candidate hypotheses.
- ordinary least squaresequivalent to · Ordinary least squares is a method for estimating model parameters by minimizing the sum of squared differences between observed values and model predictions.
- optimizationuses · Optimization is the systematic selection or modification of values, configurations, or implementations to improve a defined objective, often subject to constraints.
- algorithmkind of · An algorithm is a finite, precisely specified procedure for performing a computation, transforming inputs into outputs, or solving a problem.
- Bayesian inferencecontrasts with · Bayesian inference updates probability distributions over hypotheses or unknown quantities by combining prior information with observed evidence and a likelihood model.
Questions about maximum likelihood estimation
What is maximum likelihood estimation?
Maximum likelihood estimation is a statistical method that selects parameter values maximizing the likelihood of observed data under a specified probabilistic model.
Which Manning books explain maximum likelihood estimation?
Timeless Algorithms, The Foundational Ideas, Math and Architectures of Deep Learning, Statistics Slam Dunk, Causal AI and Grokking Statistics.
What should I know before learning maximum likelihood estimation?
Manning books that teach maximum likelihood estimation usually introduce Bayesian inference, independent variable, gradient descent and prior probability first.
What is the difference between maximum likelihood estimation and Bayesian inference?
Maximum likelihood estimation is a statistical method that selects parameter values maximizing the likelihood of observed data under a specified probabilistic model. Bayesian inference updates probability distributions over hypotheses or unknown quantities by combining prior information with observed evidence and a likelihood model.
What is maximum likelihood estimation a kind of?
Maximum likelihood estimation is a kind of algorithm. An algorithm is a finite, precisely specified procedure for performing a computation, transforming inputs into outputs, or solving a problem.
What is maximum likelihood estimation also called?
Maximum likelihood estimation is also called MLE.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




