A weighted loss function assigns different penalties to errors from different classes. It gives rare or important classes greater influence during training, helping prevent a model from ignoring them. section 3.3.2 section 6.3.2
Use weighting when class frequencies make ordinary training misleading:
- Fraud detection: 8,213 of 6.36 million transactions are fraudulent, approximately 0.13% of the dataset. section 3.3.2
- Image segmentation: increase the penalty for misclassifying forged pixels while reducing the penalty for mistakes on genuine pixels. section 6.3.2
- Tune weights carefully: values near 1 may neglect the minority class, while very large values may cause every pixel to be predicted as forged. section 6.3.2
PyTorch segmentation setup
Compute inverse class-frequency weights and pass them to CrossEntropyLoss: class_frequencies = [1.2, 1]; weights = 1.0 / torch.tensor(class_frequencies, dtype=torch.float32); loss_function = torch.nn.CrossEntropyLoss(weight=weights).
