Cross entropy loss

Q: How deeply does Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI cover cross entropy loss?

Defines the classification role and demonstrates loss-function diagnosis, but does not derive the calculation mathematically.

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

CrossEntropyLoss is a PyTorch loss-function module that combines class-logit normalization with negative log-likelihood for classification.

Cross-entropy loss trains classification models by combining class-logit normalization with negative log-likelihood. It is appropriate when the target represents categories such as 0 or 1, rather than a continuous value. section 8.2.1

Diagnosing the wrong loss

Suppose a model predicts whether an Airbnb listing's price is above or below the median. If the target remains numeric, fastai may infer regression and select MSELoss(), producing disappointing accuracy. Converting the target with merged_data['target'] = merged_data.target.astype(str) lets fastai treat it as classification; inspecting learn.loss_func then shows CrossEntropyLoss(). section 8.2.1

Listing 8.8 Statement to show the loss function used in model training

learn.loss_func                                         #1
FlattenedLoss of CrossEntropyLoss()                     #2

Inspect the configured fastai loss function and verify that classification uses CrossEntropyLoss() rather than MSELoss(). section 8.2.1

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

liveBook manning.com

Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Math and Architectures of Deep LearningA classification loss that measures the mismatch between a predicted probability distribution over classes and the target distribution.
liveBook manning.com

Grokking Machine LearningA multiclass classification loss that compares one-hot class labels with predicted class probabilities using cross-entropy.
liveBook manning.com

Inside Deep Learning: Math, Algorithms, ModelsA differentiable classification loss that measures the mismatch between predicted and target probability distributions.
liveBook manning.com

Learn Generative AI with PyTorchA loss measure that compares predicted token distributions with the target next tokens for multiclass sequence prediction.
liveBook manning.com

Graph Neural Networks in ActionCrossEntropyLoss is a PyTorch loss module that measures classification error from logits and target class labels.
liveBook manning.com

Build a Large Language Model (From Scratch)A measure of how far a language model's predicted token distribution is from the target token distribution, used to score predictions during training and evaluation.
liveBook manning.com

Deep Learning with PyTorch, Second EditionA loss value based on cross-entropy, used here as the quantity minimized during training.
liveBook manning.com

Knowledge Graphs and LLMs in ActionCross-entropy loss is a classification loss that penalizes incorrect probability assignments by comparing predicted probabilities with target labels.
liveBook manning.com

Reinforcement Learning from Human FeedbackCross-entropy is the expected negative log probability assigned by one distribution to outcomes drawn from another distribution.
liveBook manning.com

Machine Learning System DesignA classification loss that penalizes incorrect probabilistic predictions using the negative logarithm of the probability assigned to the true class.
liveBook manning.com

Deep Learning with Python, Third EditionA classification loss that measures the mismatch between predicted probability distributions and true token labels.
liveBook manning.com

How Large Language Models WorkA cross-entropy loss function measures the discrepancy between a target probability distribution and a model's predicted probability distribution.
liveBook manning.com

Build a Text-to-Image Generator (from Scratch)Categorical cross-entropy loss measures the discrepancy between predicted class probabilities and the ground-truth class in multicategory classification.
liveBook manning.com

Natural Language Processing in Action, Second EditionLog loss is a probabilistic prediction metric based on the negative logarithm of the probability assigned to the observed target.
liveBook manning.com

Deep Learning with R, Third EditionCategorical cross-entropy is a multiclass classification loss that measures the negative log-likelihood of the target class under predicted class probabilities and expects categorically encoded labels.
liveBook manning.com

Build a Reasoning Model (From Scratch)A loss function that measures the average negative log probability assigned to the correct target tokens.
liveBook manning.com

Experimentation for EngineersA loss metric that measures the discrepancy between a probability distribution predicted by a model and the observed outcome distribution.
liveBook manning.com

Grokking AI Algorithms, Second EditionCross-entropy loss is a prediction-error measure computed as the negative logarithm of the probability assigned to the correct outcome.

Compare how 19 books explain cross entropy loss →

Related concepts

classifieraddresses · A classifier is a predictive computational model that assigns inputs to one or more discrete classes or categories.
negative log-likelihoodequivalent to · Negative log-likelihood is the negative logarithm of the likelihood a probabilistic model assigns to observed data, commonly used as an objective or loss to minimize.
focal lossincludes · Focal loss is a classification loss that downweights well-classified examples and emphasizes difficult or misclassified examples, especially under class imbalance.
uncertaintymeasures · Uncertainty is incomplete knowledge or confidence about a value, state, interpretation, behavior, or outcome.
likelihoodequivalent to · A likelihood is the probability of observed data under a specified probabilistic model, regarded as a function of candidate hypotheses or parameter values.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.