Categorical cross-entropy is a multiclass classification loss that measures the negative log-likelihood of the target class under predicted class probabilities and expects categorically encoded labels.
Categorical cross-entropy is a loss used for many-class classification. Cross-entropy measures the distance between ground-truth and predicted probability distributions. section 3.6.5section 4.1.3
Choose the loss to match the task: categorical cross-entropy for many classes and binary cross-entropy for two classes. section 3.6.5
How Deep Learning with R, Third Edition covers it
How deep it goes
Defines cross-entropy as a distance between distributions and gives task-selection guidance, but no categorical-specific formula or implementation.
Examples
Contrasts many-class and two-class classification as loss-selection cases, without showing a categorical-cross-entropy code example.
What you can do after
Apply it as the loss for a many-class classification task.
How it connects
Links categorical cross-entropy to multiclass classification and contrasts it with binary cross-entropy for two-class classification.
Math and Architectures of Deep LearningA classification loss that measures the mismatch between a predicted probability distribution over classes and the target distribution.
Learn Generative AI with PyTorchA loss measure that compares predicted token distributions with the target next tokens for multiclass sequence prediction.
Build a Large Language Model (From Scratch)A measure of how far a language model's predicted token distribution is from the target token distribution, used to score predictions during training and evaluation.
Knowledge Graphs and LLMs in ActionCross-entropy loss is a classification loss that penalizes incorrect probability assignments by comparing predicted probabilities with target labels.
Reinforcement Learning from Human FeedbackCross-entropy is the expected negative log probability assigned by one distribution to outcomes drawn from another distribution.
Machine Learning System DesignA classification loss that penalizes incorrect probabilistic predictions using the negative logarithm of the probability assigned to the true class.
How Large Language Models WorkA cross-entropy loss function measures the discrepancy between a target probability distribution and a model's predicted probability distribution.
Build a Text-to-Image Generator (from Scratch)Categorical cross-entropy loss measures the discrepancy between predicted class probabilities and the ground-truth class in multicategory classification.
Experimentation for EngineersA loss metric that measures the discrepancy between a probability distribution predicted by a model and the observed outcome distribution.
Grokking AI Algorithms, Second EditionCross-entropy loss is a prediction-error measure computed as the negative logarithm of the probability assigned to the correct outcome.
classifieraddresses · A classifier is a predictive computational model that assigns inputs to one or more discrete classes or categories.
negative log-likelihoodequivalent to · Negative log-likelihood is the negative logarithm of the likelihood a probabilistic model assigns to observed data, commonly used as an objective or loss to minimize.
focal lossincludes · Focal loss is a classification loss that downweights well-classified examples and emphasizes difficult or misclassified examples, especially under class imbalance.
uncertaintymeasures · Uncertainty is incomplete knowledge or confidence about a value, state, interpretation, behavior, or outcome.
likelihoodequivalent to · A likelihood is the probability of observed data under a specified probabilistic model, regarded as a function of candidate hypotheses or parameter values.
This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.
