Focal loss modifies classification loss so easy, well-predicted examples contribute less, while hard examples receive more emphasis. This makes it useful for data-imbalance problems. section 9.1.7 section 9.4
Let y be the predicted probability and ȳ the binary ground-truth label. Define y_t = ȳy + (1 − ȳ)(1 − y), the probability assigned to the true class. The focal-loss expression is −(1 − y_t)^γ log(y_t). The focusing parameter γ controls how strongly easy examples are downweighted; γ = 2 is a typical choice. As the true-class probability approaches one, the loss is increasingly subdued and eventually flattens near zero. section 9.1.7
The weighting follows the ground-truth class:
- For ȳ = 1, use (1 − y)^γ to reduce the contribution of good predictions and emphasize bad predictions. section 9.1.7
- For ȳ = 0, use y^γ for the analogous weighting of the predicted probability. section 9.1.7
PyTorch implementation
A compact implementation first selects the true-class probability, then applies the focal factor and logarithmic loss: y_t = (y_gt * y) + ((1 - y_gt) * (1 - y)); loss = -torch.pow((1 - y_t), gamma) * torch.log(y_t). section 9.1.7
Equation 9.7
Use the displayed equation to inspect the standard focal-loss form. section 9.1.7
Listing 9.6 PyTorch code for focal loss
def focal_loss(y, y_gt, gamma):
y_t = (y_gt * y) + ((1 - y_gt) * (1 - y)) #1
loss = -1 * torch.pow((1 - y_t), gamma) * torch.log(y_t)
return loss
Use the executable listing to connect the binary formula with PyTorch operations. section 9.1.7
