Optimization adjusts a model’s weights to improve its performance on training data. A loss function compares predictions with targets; an optimizer uses that feedback to change the weights in a direction that lowers the loss. Repeating this process across batches turns initially random weights into a trained network. section 1.6 section 5.1
Train a digit classifier
In R with Keras, choose the optimizer when configuring the model, then call fit() to train it: library(keras3) .[.[train_images, train_labels], .[test_images, test_labels]] <- dataset_mnist() train_images <- array_reshape(train_images, c(60000, 28 * 28)) / 255 test_images <- array_reshape(test_images, c(10000, 28 * 28)) / 255 model <- keras_model_sequential() |> layer_dense(512, activation = "relu") |> layer_dense(10, activation = "softmax") model |> compile( optimizer = "adam", loss = "sparse_categorical_crossentropy", metrics = "accuracy" ) fit(model, train_images, train_labels, epochs = 5, batch_size = 128) evaluate(model, test_images, test_labels) Here, the loss supplies the signal to minimize, Adam specifies how weights are updated, and accuracy is monitored rather than directly optimized. Each epoch passes through the training data. section 2.1 section 2.5 section 3.6.4
To configure optimization, keep three choices separate:
- Choose a loss function suited to the task: it defines what training minimizes. section 3.6.4
- Choose an optimizer to govern the updates. Keras accepts a shortcut such as "adam" or an optimizer object; an object can set options such as learning_rate. section 2.5 section 3.6.4
- Choose metrics to monitor performance. Unlike the loss, they are not directly optimized. section 3.6.4
Lower training loss is not the whole goal. Continued fitting can improve training performance while performance on unseen data worsens—overfitting. Monitor training and validation results to spot that divergence. When checking whether a model has enough capacity, adding layers, enlarging layers, or training longer can deliberately expose the point where validation performance begins to degrade. section 5.1 section 6.2.4
