An objective function, also called a loss or cost function, turns prediction errors into a value used to train a model. Its choice determines which mistakes the model works hardest to avoid. Changing the loss can therefore change the system’s behavior even when its features, data, and architecture stay the same. section 5.1
Start with the real decision the system must support, not the loss used on a previous project. A training loss generally needs to be continuous and differentiable for gradient-based optimization. A business metric may express the goal more directly but may not work as a loss: every loss can serve as a metric, whereas not every metric can train a model. section 5.0 section 5.1
When choosing an objective, check:
- Which errors matter most to the product, and whether the proposed loss rewards the right behavior. section 5.1 section 5.3.3
- Whether difficult cases need more attention. For imbalanced classification, focal loss down-weights easy examples so training focuses more on hard ones. section 5.1.1
- Whether improvements in loss also improve the metrics that matter after deployment. If they do not, revisit the objective and metric hierarchy. section 9.1.3
Forecasts for stock decisions
A retailer may care about maintaining a service level rather than minimizing average forecast error. Quantile forecasts can support stock-control decisions; an appropriate quantile loss can then align training with a quantile metric. Weights can give particular products or groups more attention. section 5.3.1
A falling loss is not always success
A spam classifier trained with log loss can improve its binary classification result without adequately addressing what users experience. High-volume bursts from one sender illustrate why the operational goal may require a custom loss, a hierarchy of metrics, and continued experimentation. section 9.1.3
Figure 9.6 Loss is decreasing for training but not validation, reflecting potential overfit.
Compare training and validation loss curves: falling training loss alongside rising validation loss signals potential overfitting, not a reason to keep optimizing unchanged. section 9.1.3
Treat loss curves as diagnostic evidence. An unexpected loss value warrants a single-batch sanity check, including any scaling transformations; a declining training loss with rising validation loss calls for less model capacity or more regularization. Neither check replaces choosing metrics that reflect the goal. section 9.1.3
