liveBook · Manning

Log-likelihood

As explained in Timeless Algorithms, The Foundational Ideas

A log-likelihood is the logarithm of a likelihood function, converting products of probabilities into sums for analysis and optimization.

A likelihood multiplies the probabilities or densities assigned to observed data under a model. Its logarithm, the log-likelihood, turns that product into a sum. Because the natural logarithm is strictly increasing, the parameter values that maximize the likelihood also maximize the log-likelihood. section 3.1.4

l(μ)=log⁡L(μ)

For a normal model with known variance, maximizing this expression leads to the sample mean as the maximum-likelihood estimate. section 3.1.4

The transformation makes optimization easier to express: products become sums, and terms constant with respect to the parameter can be set aside. In logistic regression, maximizing the log-likelihood requires iterative numerical optimization rather than a closed-form solution. section 3.1.4 section 3.3.2

Log-likelihood also connects probability models to familiar fitting methods. With normally distributed regression errors and fixed variance, maximizing it yields the same coefficients as ordinary least squares, which minimizes squared residuals. section 3.4.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Derives the normal-mean result, gives the logistic-regression objective, and notes that its optimization requires iteration.
Examples
Uses a normal sample mean, logistic-regression coefficients, and linear regression with normally distributed errors.
What it assumes
Assumes a specified probability model and, in the worked likelihoods, independent observations.
What you can do after
Understand why maximizing a log-likelihood can simplify estimation and coincide with minimizing squared residuals.
How it connects
Connects maximum likelihood estimation with logistic regression and ordinary least squares.

Also explained in

Compare how 4 books explain log-likelihood →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.