How deeply does Deep Learning with PyTorch, Second Edition cover negative log-likelihood?

Defines the formula, explains probability sensitivity, gives calculation steps, and demonstrates PyTorch's log-probability convention.

Negative log-likelihood

As explained in Deep Learning with PyTorch, Second Edition

A loss function that penalizes low predicted probability assigned to the correct class by taking the negative logarithm of that probability and summing over samples.

Negative log-likelihood (NLL) is a classification loss that penalizes assigning low probability to the correct class. It takes the negative logarithm of that probability and sums across samples. section 7.2.4

For each supervised sample:

Run the model forward, convert output scores into probabilities, select the probability assigned to the ground-truth class, take its logarithm, negate it, and add it to the loss. section 7.2.4
Low assigned probabilities produce sharply increasing loss, while probabilities above 0.5 reduce the loss more gradually. section 7.2.4
PyTorch's nn.NLLLoss expects log probabilities rather than probabilities, typically produced with nn.LogSoftmax. section 7.2.4

PyTorch image classification

A two-class CIFAR model can end with nn.LogSoftmax(dim=1), then use loss = nn.NLLLoss(). Passing the model output and the ground-truth label returns the sample loss.

How Deep Learning with PyTorch, Second Edition covers it

Cover of Deep Learning with PyTorch, Second Edition

liveBook manning.com

Read Deep Learning with PyTorch, Second Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Optimization Algorithms: AI techniques for design, planning, and control problemsNegative log likelihood is a loss equal to the negative logarithm of the probability assigned to the true label.
liveBook manning.com

Reinforcement Learning from Human FeedbackNegative log-likelihood is the negation of a model's log-likelihood, used as a loss for probabilistic predictions.
liveBook manning.com

Natural Language Processing in Action, Second EditionNegative log likelihood is the negative of the logarithm of a model-assigned likelihood, commonly minimized as a probabilistic prediction loss.
liveBook manning.com

Causal AIThe negative log-likelihood loss function is the negative logarithm of the likelihood and is minimized to estimate parameters.
liveBook manning.com

Machine Learning Algorithms in DepthA negative log likelihood is a loss equal to the negative logarithm of the likelihood assigned to observed outcomes by a probabilistic model.
liveBook manning.com

Ensemble Methods for Machine LearningNegative log-likelihood is the negative logarithm of the likelihood assigned by a probabilistic model to observed data.

Compare how 7 books explain negative log-likelihood →

Related concepts

cross entropy lossequivalent to · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
mean squared errorequivalent to · Mean squared error is the arithmetic mean of the squared differences between corresponding predicted and target numerical values.
DPO lossused by · The DPO loss is a preference-training objective that optimizes a policy to assign higher relative likelihood to preferred responses than to rejected responses compared with a reference policy.
NLLLossused by · NLLLoss is a PyTorch loss function that computes negative log-likelihood loss from log-probability outputs and target classes.
mixture density networkused by · A mixture density network is a neural network that predicts the parameters of a conditional mixture probability distribution, optionally at each sequence step.
variational lossy autoencodermeasures · A variational lossy autoencoder is an autoencoder variant that intentionally constrains its latent representation so reconstruction preserves selected information rather than every input detail.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with PyTorch, Second Edition. The section links point to the parts of the book it draws on.