Fine-tuning continues training a pretrained model for a new task. Rather than learning every representation from scratch, it adjusts learned representations to fit the new problem. It complements feature extraction, which reuses a model with its base frozen. section 8.3.2 section 8.4
Dogs versus cats with Xception
Start with an ImageNet-pretrained Xception convolutional base and add a dogs-versus-cats classifier. Freeze the base and train the new classifier first. Then unfreeze the base and train both parts together. This order matters: a randomly initialized classifier can produce an error signal large enough to damage the base’s learned representations. During fine-tuning, keep BatchNormalization layers frozen. section 8.3 section 8.3.2
Listing 8.30 Fine-tuning the model
model |> compile(
loss = "binary_crossentropy",
optimizer = optimizer_adam(learning_rate = 1e-5),
metrics = "accuracy"
)
callbacks <- list(
callback_model_checkpoint(
filepath = "fine_tuning.keras",
save_best_only = TRUE,
monitor = "val_loss"
)
)
history <- model |> fit(
augmented_train_dataset,
epochs = 30,
validation_data = validation_dataset,
callbacks = callbacks
)
The R/Keras training step recompiles the model with Adam at a learning rate of 1e-5, uses augmented training images, and saves the checkpoint with the lowest validation loss. section 8.3.2
Unfreezing the entire base is not the only choice. For a large pretrained model, you can leave lower layers frozen and fine-tune only upper layers: lower layers encode more-generic features, while upper layers encode features more likely to need repurposing. In the dogs-versus-cats run, the saved model reaches 0.986 test accuracy. section 8.3.2
IMDb reviews with RoBERTa
Fine-tuning also works for text classification. Tokenize IMDb reviews with RoBERTa’s tokenizer and preserve its pretrained start, end, and padding-token arrangement. Add a binary classification head to the backbone; here, the head takes the first token’s contextual representation rather than averaging all token vectors. Training the classifier with Adam at 5e-5 reaches about 93% accuracy after one epoch, at a higher computational cost than a simple bigram classifier. section 15.4.3 section 15.4.4
Listing 15.22 Extending the base RoBERTa model for classification
inputs <- backbone$input outputs <- inputs |> backbone() |> op_subset(, 1, ) |> #1 layer_dropout(0.1) |> layer_dense(768, activation = "relu") |> layer_dropout(0.1) |> layer_dense(1, activation = "sigmoid") classifier <- keras_model(inputs, outputs)
The classification-head listing shows how the first token’s representation feeds dropout, a Dense layer, and a sigmoid output. section 15.4.4
For a generative model, instruction fine-tuning uses instruction–response pairs instead of class labels. Join each instruction and desired response into a consistently marked sequence, then continue training with next-token prediction. This nudges an existing language model toward the requested tone and content; it does not teach language from zero. section 16.3.2
Instruction-following with Gemma
A Gemma exercise formats handwritten instructions and responses from the Databricks Dolly dataset using markers such as [instruction] and [response]. This supervised fine-tuning approach depends on examples people have written; collecting them can be slow and expensive. Reinforcement learning with human feedback is presented as a further way to optimize for preferred responses. section 16.3.2 section 16.4.1
