liveBook · Manning

Negative log-likelihood

As explained in Timeless Algorithms, The Foundational Ideas

A negative log-likelihood is the negative logarithm of a likelihood, commonly minimized as an objective equivalent to maximizing likelihood.

For observed data D and model parameters θ, the negative log-likelihood is −log L(θ; D). Minimizing it is equivalent to choosing parameters that make the observed data most probable under an assumed probability model. Neural networks commonly optimize this objective with gradient descent rather than closed-form estimation. section 3.5

The task determines which likelihood-based loss is used:

  • Classification commonly uses cross-entropy, the negative log-likelihood for a categorical probability model. section 3.5
  • Regression commonly uses mean squared error, corresponding to normally distributed errors. section 3.5

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains likelihood equivalence and task-specific losses, but defers detailed neural-network treatment.
Examples
Uses classification with cross-entropy and regression with mean squared error as likelihood-based objectives.
What you can do after
Understand common neural-network losses as objectives that maximize observed-data likelihood under specified probability models.
How it connects
Links maximum likelihood, cross-entropy, mean squared error, categorical probability models, normal errors, and gradient descent.

Also explained in

Compare how 8 books explain negative log-likelihood →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.