Hyperparameter tuning tests model configurations instead of relying on the first plausible choice. Hyperparameters include layer sizes, activations, dropout, and optimizer choices; unlike model parameters, they are not learned by backpropagation. Each candidate model is trained on training data and compared using validation performance. section 18.1.1
Listing 18.1 KerasTuner model-building function
build_model <- function(hp, num_classes = 10) {
units <- hp$Int( #1
name = "units", #1
min_value = 16L, #1
max_value = 64L, #1
step = 16L) #1
model <- keras_model_sequential() |> #1
layer_dense(units, activation = "relu") |>
layer_dense(num_classes, activation = "softmax")
optimizer <- hp$Choice(name = "optimizer", #2
values = c("rmsprop", "adam"))
model |> compile(optimizer = optimizer,
loss = "sparse_categorical_crossentropy",
metrics = "accuracy")
model #3
}
Start with a model-building function. Here, KerasTuner chooses the number of units from 16 to 64 and the optimizer from RMSprop or Adam. The function builds and compiles a fresh model for each choice. section 18.1.1
tuner <- kt$BayesianOptimization( build_model, #1 objective = "val_accuracy", #2 max_trials = 20L, #3 executions_per_trial = 2L, #4 directory = "mnist_kt_test", #5 overwrite = TRUE #6 )
A Bayesian-optimization tuner searches for high validation accuracy. It allows up to 20 trials, runs each trial twice, and stores results in a named directory. section 18.1.1
.[.[x_train_full, y_train_full], .[x_test, y_test]] <- dataset_mnist() #1 x_train_full <- x_train_full |> array_reshape(c(-1, 28 * 28)) #1 x_train_full <- x_train_full / 255 #1 x_test <- x_test |> array_reshape(c(-1, 28 * 28)) #1 x_test <- x_test / 255 #1 num_val_samples <- 10000 #2 val_i <- seq_len(num_val_samples) #2 x_val <- x_train_full[val_i, ] #2 x_train <- x_train_full[-val_i, ] #2 y_val <- y_train_full[val_i] |> as.matrix() #2 y_train <- y_train_full[-val_i] |> as.matrix() #2 callbacks <- list( #3 callback_early_stopping(monitor = "val_loss", patience = 5) #3 ) tuner$search( #4 x_train, y_train, batch_size = 128L, epochs = 100L, validation_data = list(x_val, y_val), callbacks = callbacks, verbose = 2L )
In the MNIST example, images are flattened and scaled, then 10,000 training examples are set aside for validation. The tuner trains candidate models with early stopping and checks them against that validation split—not the test set. section 18.1.1
After searching, retrieve the best configurations. For a final model, determine a suitable training duration, rebuild it, train on the combined training and validation data, and evaluate once on the test data. Repeatedly adapting choices to validation results can overfit the validation process; a much worse test result may call for a more reliable evaluation protocol. section 18.1.1 section 6.2.5
