Focal loss
Focal loss is a classification loss that downweights well-classified examples and emphasizes difficult or misclassified examples, especially under class imbalance.
Machine Learning System Design
A classification loss that down-weights well-classified high-confidence examples so training emphasizes difficult examples.
- How it is used
- Addresses class imbalance by shifting training emphasis from easy, high-confidence examples toward difficult examples.
- How deep it goes
- Defines dynamic scaling and practical tradeoffs but stops short of mathematical derivation or implementation details.
- Examples
- Uses object detection, imbalanced clicks and purchases, and a search-engine diversity objective as concrete applications.
- What you can do after
- Apply focal loss when imbalanced classification makes ordinary cross-entropy insufficiently attentive to difficult examples.
Read how Machine Learning System Design explains focal loss →
Math and Architectures of Deep Learning
A modified classification loss that downweights easy examples and emphasizes hard examples, especially under class imbalance.
- How it is used
- Uses a classification loss to address data imbalance by emphasizing hard examples.
- How deep it goes
- Defines binary weighting mathematically and supplies PyTorch code, but does not develop implementation beyond the loss calculation.
- Examples
- Uses γ = 2, true-class probability curves, binary cases, and a PyTorch focal_loss function.
- What you can do after
- Apply focal loss through its binary formula or the supplied PyTorch implementation.
Read how Math and Architectures of Deep Learning explains focal loss →
Ensemble Methods for Machine Learning
Focal loss is a classification loss that down-weights well-classified examples and emphasizes poorly classified examples, especially under class imbalance.
- How it is used
- Targets classification problems where class imbalance makes easy examples overwhelm learning.
- How deep it goes
- Defines the modulation mechanism, compares gamma values, and implements custom LightGBM metric and objective functions.
- Examples
- Uses dense object-detection windows, gamma-based loss curves, custom Python functions, and LightGBM validation predictions.
- What you can do after
- Apply it via LightGBM by defining focal-loss derivatives, training a classifier, and thresholding predicted probabilities.
Read how Ensemble Methods for Machine Learning explains focal loss →
What to know first
- cross entropy lossCross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
- object detectionObject detection is a computer-vision task that identifies object instances in visual data, assigns them categories, and estimates their locations.
- recallRecall is the proportion of all relevant or actual positive items that a system successfully retrieves or correctly identifies.
Related concepts
- cross entropy losskind of · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
- LightGBMused by · LightGBM is an open-source gradient-boosting framework for efficiently training tree-based machine-learning models.
- class imbalanceaddresses · Class imbalance is a condition in a classification dataset where classes have substantially unequal numbers of examples.
- fine-tuningused by · Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- object detectionaddresses · Object detection is a computer-vision task that identifies object instances in visual data, assigns them categories, and estimates their locations.
Questions about focal loss
What is focal loss?
Focal loss is a classification loss that downweights well-classified examples and emphasizes difficult or misclassified examples, especially under class imbalance.
Which Manning books explain focal loss?
Machine Learning System Design, Math and Architectures of Deep Learning and Ensemble Methods for Machine Learning.
What should I know before learning focal loss?
Manning books that teach focal loss usually introduce cross entropy loss, object detection and recall first.
What is focal loss a kind of?
Focal loss is a kind of cross entropy loss. Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


