Focal loss
Focal loss is a classification loss that downweights well-classified examples and emphasizes difficult or misclassified examples, especially under class imbalance.
Machine Learning System Design
A classification loss that down-weights well-classified high-confidence examples so training emphasizes difficult examples.
- How it is used
- Addresses class imbalance by shifting training emphasis from easy, high-confidence examples toward difficult examples.
- How deep it goes
- Defines dynamic scaling and practical tradeoffs but stops short of mathematical derivation or implementation details.
- Examples
- Uses object detection, imbalanced clicks and purchases, and a search-engine diversity objective as concrete applications.
- What you can do after
- Apply focal loss when imbalanced classification makes ordinary cross-entropy insufficiently attentive to difficult examples.
Read how Machine Learning System Design explains focal loss →
Math and Architectures of Deep Learning
A modified classification loss that downweights easy examples and emphasizes hard examples, especially under class imbalance.
- How it is used
- Uses a classification loss to address data imbalance by emphasizing hard examples.
- How deep it goes
- Defines binary weighting mathematically and supplies PyTorch code, but does not develop implementation beyond the loss calculation.
- Examples
- Uses γ = 2, true-class probability curves, binary cases, and a PyTorch focal_loss function.
- What you can do after
- Apply focal loss through its binary formula or the supplied PyTorch implementation.
Read how Math and Architectures of Deep Learning explains focal loss →
Ensemble Methods for Machine Learning
Focal loss is a classification loss that down-weights well-classified examples and emphasizes poorly classified examples, especially under class imbalance.
- How it is used
- Targets classification problems where class imbalance makes easy examples overwhelm learning.
- How deep it goes
- Defines the modulation mechanism, compares gamma values, and implements custom LightGBM metric and objective functions.
- Examples
- Uses dense object-detection windows, gamma-based loss curves, custom Python functions, and LightGBM validation predictions.
- What you can do after
- Apply it via LightGBM by defining focal-loss derivatives, training a classifier, and thresholding predicted probabilities.
Read how Ensemble Methods for Machine Learning explains focal loss →
Related concepts
- cross entropy losskind of · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
- LightGBMused by · LightGBM is an open-source gradient-boosting framework for efficiently training tree-based machine-learning models.
- class imbalanceaddresses · Class imbalance is a condition in a classification dataset where classes have substantially unequal numbers of examples.
- fine-tuningused by · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
- object detectionaddresses · Object detection is a computer-vision task that identifies object instances in visual data, assigns them categories, and estimates their locations.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.