For observed data D and model parameters θ, the negative log-likelihood is −log L(θ; D). Minimizing it is equivalent to choosing parameters that make the observed data most probable under an assumed probability model. Neural networks commonly optimize this objective with gradient descent rather than closed-form estimation. section 3.5
The task determines which likelihood-based loss is used:
- Classification commonly uses cross-entropy, the negative log-likelihood for a categorical probability model. section 3.5
- Regression commonly uses mean squared error, corresponding to normally distributed errors. section 3.5







