liveBook · Manning

Negative log-likelihood

Also called NLL

Negative log-likelihood is the negative logarithm of the likelihood a probabilistic model assigns to observed data, commonly used as an objective or loss to minimize.

How 8 Manning books explain it

Optimization Algorithms: AI techniques for design, planning, and control problems

Negative log likelihood is a loss equal to the negative logarithm of the probability assigned to the true label.

How it is used
Uses probability assigned to the correct class as a loss for training multiclass neural networks.
How deep it goes
Defines NLL and explains its behavior, then demonstrates batch computation but stops short of deriving optimization details.
Examples
Demonstrates NLL inside NumPy forward propagation for a neural network trained on the Penguins classification dataset.
What you can do after
Apply it via NumPy to compute and average classification loss from logits, probabilities, and true labels.

Read how Optimization Algorithms: AI techniques for design, planning, and control problems explains negative log-likelihood →

Reinforcement Learning from Human Feedback

Negative log-likelihood is the negation of a model's log-likelihood, used as a loss for probabilistic predictions.

How it is used
Uses negative log-likelihood as a preference-training loss and as a language-modeling regularizer.
How deep it goes
Derives preference losses and equivalent forms, then expands a weighted, length-normalized NLL penalty.
Examples
Works through chosen-versus-rejected reward scores and a DPO objective with a winning-completion penalty.
What you can do after
Evaluate and understand how preference likelihood and absolute text likelihood contribute to training objectives.

Read how Reinforcement Learning from Human Feedback explains negative log-likelihood →

Deep Learning with PyTorch, Second Edition

A loss function that penalizes low predicted probability assigned to the correct class by taking the negative logarithm of that probability and summing over samples.

How it is used
Uses predicted class likelihoods to train a supervised image classifier.
How deep it goes
Defines the formula, explains probability sensitivity, gives calculation steps, and demonstrates PyTorch's log-probability convention.
Examples
Demonstrates CIFAR bird-versus-airplane classification with nn.LogSoftmax and nn.NLLLoss.
What you can do after
Apply it via PyTorch by producing log probabilities and passing them with target class labels to nn.NLLLoss.

Read how Deep Learning with PyTorch, Second Edition explains negative log-likelihood →

Natural Language Processing in Action, Second Edition

Negative log likelihood is the negative of the logarithm of a model-assigned likelihood, commonly minimized as a probabilistic prediction loss.

How it is used
Uses NLLLoss to train an RNN and translate prediction scores into communicable confidence estimates.
How deep it goes
Explains logarithm inversion and likelihood interpretation, then stops short of a deeper derivation of the loss.
Examples
Uses a nationality-prediction model, the name Khalid, ranked log-loss values, and exponentiated likelihoods.
What you can do after
Evaluate/understand model confidence by converting log-loss outputs into likelihoods and inspecting ranked predictions.

Read how Natural Language Processing in Action, Second Edition explains negative log-likelihood →

Causal AI

The negative log-likelihood loss function is the negative logarithm of the likelihood and is minimized to estimate parameters.

How it is used
Uses likelihood-based estimation as a machine-learning loss optimized for model parameters.
How deep it goes
Defines the likelihood transformation and optimization role, then connects it to numerical optimization and regularization.
Examples
Uses linear regression, neural networks, and L2 regularization as estimation and optimization examples.
What you can do after
Apply it via numerical optimization to estimate parameters when likelihood maximization lacks a closed-form solution.

Read how Causal AI explains negative log-likelihood →

Machine Learning Algorithms in Depth

A negative log likelihood is a loss equal to the negative logarithm of the likelihood assigned to observed outcomes by a probabilistic model.

How it is used
Uses likelihood-based loss to train probabilistic classifiers and neural networks that predict continuous probability densities.
How deep it goes
Derives gradients and implements SGD for logistic regression, then constructs constrained Gaussian-mixture loss, stopping short of broader likelihood theory.
Examples
Uses binary logistic regression with synthetic two-class data and a Gaussian mixture density network trained on synthetic multivariate data.
What you can do after
Builds NLL objectives from scratch, optimizes logistic regression with SGD, and applies NLL in a Keras/TensorFlow mixture density network.

Read how Machine Learning Algorithms in Depth explains negative log-likelihood →

Ensemble Methods for Machine Learning

Negative log-likelihood is the negative logarithm of the likelihood assigned by a probabilistic model to observed data.

How it is used
Connects assumed residual distributions with regression losses and their optimization behavior.
How deep it goes
Defines the optimization role and compares distribution-linked losses, stopping short of detailed derivations.
Examples
Uses Gaussian, Laplacian, and Huber losses for continuous-valued targets, alongside comparisons across target types.
What you can do after
Evaluate how distribution assumptions change loss penalties, differentiability, outlier sensitivity, and boosting suitability.

Read how Ensemble Methods for Machine Learning explains negative log-likelihood →

Timeless Algorithms, The Foundational Ideas

A negative log-likelihood is the negative logarithm of a likelihood, commonly minimized as an objective equivalent to maximizing likelihood.

How it is used
Connects neural-network loss minimization to maximum-likelihood estimation under an assumed probability model.
How deep it goes
Explains likelihood equivalence and task-specific losses, but defers detailed neural-network treatment.
Examples
Uses classification with cross-entropy and regression with mean squared error as likelihood-based objectives.
What you can do after
Understand common neural-network losses as objectives that maximize observed-data likelihood under specified probability models.

Read how Timeless Algorithms, The Foundational Ideas explains negative log-likelihood →

Also covered in

What to know first

Related concepts

Questions about negative log-likelihood

What is negative log-likelihood?

Negative log-likelihood is the negative logarithm of the likelihood a probabilistic model assigns to observed data, commonly used as an objective or loss to minimize.

Which Manning books explain negative log-likelihood?

Optimization Algorithms: AI techniques for design, planning, and control problems, Reinforcement Learning from Human Feedback, Deep Learning with PyTorch, Second Edition, Natural Language Processing in Action, Second Edition, Causal AI, Machine Learning Algorithms in Depth, Ensemble Methods for Machine Learning and Timeless Algorithms, The Foundational Ideas.

What should I know before learning negative log-likelihood?

Manning books that teach negative log-likelihood usually introduce language model, closed form, fine-tuning and logarithm first.

What is negative log-likelihood also called?

Negative log-likelihood is also called NLL.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.