With few data points, a single validation set may be so small that the score changes substantially depending on which examples it contains. K-fold cross-validation addresses this by dividing the available data into K disjoint partitions, or folds—typically four or five—and giving each fold a turn as the validation set. section 4.3.4
For each fold:
- Set that partition aside for validation and use the other K − 1 partitions for training. section 4.3.4
- Create a fresh, identical model, train it, and record its validation score. section 4.3.4
- Average the K scores rather than trusting any one split. section 4.3.4
Four folds in practice
With K = 4, one run produced validation scores of 0.298, 0.349, 0.232, and 0.305. Their mean was 0.296. That spread shows why the choice of validation fold matters; the mean gives a more reliable metric than a single score. Averaging each fold’s validation MAE at every epoch can also help choose a training duration before fitting a final model on all the training data. section 4.3.4
Listing 4.28 K-fold validation
k = 4 num_val_samples = len(x_train) // k num_epochs = 50 all_scores = [] for i in range(k): print(f"Processing fold #{i + 1}") fold_x_val = x_train[i * num_val_samples : (i + 1) * num_val_samples] #1 fold_y_val = y_train[i * num_val_samples : (i + 1) * num_val_samples] fold_x_train = np.concatenate( #2 [x_train[: i * num_val_samples], x_train[(i + 1) * num_val_samples :]], #2 axis=0, #2 ) #2 fold_y_train = np.concatenate( [y_train[: i * num_val_samples], y_train[(i + 1) * num_val_samples :]], axis=0, ) model = get_model() #3 model.fit( #4 fold_x_train, #4 fold_y_train, #4 epochs=num_epochs, #4 batch_size=16, #4 verbose=0, #4 ) #4 scores = model.evaluate(fold_x_val, fold_y_val, verbose=0) #5 val_loss, val_mae = scores all_scores.append(val_mae)
The K-fold validation listing shows how array slices select one fold and concatenation assembles the remaining training data. section 4.3.4
