An objective function, also called a loss or cost function, measures how far a model’s prediction is from its target. Its numerical result is the feedback signal for learning: the optimizer adjusts the model’s weights to reduce the loss. section 1.6
Train a digit classifier
For a model that predicts a probability for each digit, compile it with `model |> compile(optimizer = "adam", loss = "sparse_categorical_crossentropy", metrics = "accuracy")`, then train with `fit(model, train_images, train_labels, epochs = 5, batch_size = 128)`. The loss guides weight updates; accuracy reports the fraction of correctly classified images. section 2.1 section 2.5
Choose the loss for the job, not merely for a low number. Binary cross-entropy suits two-class classification; categorical cross-entropy suits multiclass classification; mean squared error suits regression. A poorly matched objective can reward shortcuts rather than the behavior you intended. section 3.6.5 section 6.2.3
Table 6.1 Which loss function, last-layer activation, and metrics to use for different tasks
|
Task
|
Last-layer activation
|
Loss function
|
Metrics
|
|---|---|---|---|
|
Binary classification
|
Sigmoid
|
Binary cross-entropy
|
Binary accuracy, ROC AUC
|
|
Multiclass, single-label classification
|
Softmax
|
Categorical cross-entropy
|
Categorical accuracy, top-
k categorical accuracy, ROC AUC
|
|
Multiclass, multilabel classification
|
Sigmoid
|
Binary cross-entropy
|
Binary accuracy, ROC AUC
|
|
Regression
|
None
|
Mean squared error
|
Mean absolute error
|
Compare common tasks alongside their last-layer activations, losses, and monitoring metrics. section 6.2.3
Loss and metrics have different jobs. Training minimizes a differentiable loss computable on a mini-batch; a metric such as accuracy or ROC AUC measures a result you care about but need not be directly optimized. For a classifier, cross-entropy can serve as a proxy when ROC AUC cannot be optimized directly. Watch training and validation loss together: worsening validation performance while training continues is a sign of overfitting. section 3.6.4 section 6.2.3 section 6.2.4
When a built-in loss is not enough
An object detector can need a custom bounding-box loss. Its confidence target can use intersection over union for cells containing an object and zero for empty cells. Scaling placement errors up and empty-cell confidence errors down keeps the latter from overwhelming training. section 12.2.4
