Validation loss is the loss computed on data withheld from training. It estimates whether a model has learned patterns useful for unseen examples rather than merely memorizing training samples. Keep training and validation data strictly separate; otherwise, the estimate is flawed. section 3.6.7
Spotting overfitting in image classification
During ConvNet training, training loss continues to decrease while validation loss reaches its minimum after about 10 epochs and then stalls. Configure ModelCheckpoint with save_best_only=True and monitor="val_loss" so the saved model corresponds to the lowest validation loss, then reload it before evaluation. section 8.2.4
Use validation loss in three practical ways:
- Pass held-out arrays or a Dataset through validation_data in fit() to monitor loss and other metrics during training. section 3.6.7
- Call evaluate() after training to compute validation loss in batches; it is returned first, followed by configured validation metrics. section 3.6.7
- Compare training and validation curves: a widening difference, especially with rising or stalled validation loss, signals overfitting. section 8.2.4
Listing 3.35 Using the validation data argument
model = keras.Sequential([keras.layers.Dense(1)]) model.compile( optimizer=keras.optimizers.RMSprop(learning_rate=0.1), loss=keras.losses.MeanSquaredError(), metrics=[keras.metrics.BinaryAccuracy()], ) indices_permutation = np.random.permutation(len(inputs)) #1 shuffled_inputs = inputs[indices_permutation] #1 shuffled_targets = targets[indices_permutation] #1 num_validation_samples = int(0.3 * len(inputs)) #2 val_inputs = shuffled_inputs[:num_validation_samples] #2 val_targets = shuffled_targets[:num_validation_samples] #2 training_inputs = shuffled_inputs[num_validation_samples:] #2 training_targets = shuffled_targets[num_validation_samples:] #2 model.fit( training_inputs, #3 training_targets, #3 epochs=5, batch_size=16, validation_data=(val_inputs, val_targets), #4 )
Inspect the complete Keras workflow for shuffling data, reserving validation samples, fitting with validation_data, and evaluating the held-out split. section 3.6.7
