Chapter 6: Fine-tuning for classification

Introduction

The chapter introduces classification fine-tuning as adapting a pretrained LLM to a specific task, using spam detection as its example.

• Classification fine-tuning purpose: Explain that a pretrained LLM can be fine-tuned for a specific target task such as classifying text messages as spam or not spam.
  Why it matters: This identifies what the pretrained model is being adapted to do before the chapter explains how.

Key terms: pretrained LLM, fine-tuning for classification, spam, not spam

6.1 Different categories of fine-tuning

This section contrasts instruction and classification fine-tuning and explains when each approach is appropriate.

• Fine-tuning task distinctions: Distinguish instruction fine-tuning from classification fine-tuning by describing what each trains a model to do and what a classification model can predict.
  Why it matters: The distinction clarifies why a spam classifier and a model that follows varied prompts serve different purposes.
• Choosing a fine-tuning approach: Justify choosing instruction fine-tuning for varied instruction-driven tasks or classification fine-tuning for precise predefined categories, including the relative data and compute demands.
  Why it matters: Choosing the approach means balancing the range of tasks needed against the available data and computation.

Key terms: instruction fine-tuning, classification fine-tuning, class labels, predefined classes, computational resources

6.2 Preparing the dataset

This section prepares an SMS spam dataset for classification fine-tuning by balancing its classes, encoding its labels, and splitting and saving the data.

• SMS dataset and class balance: Explain how the SMS dataset is loaded and why the author balances its spam and non-spam messages by undersampling.
  Why it matters: Balancing the classes establishes the dataset used for the subsequent classification steps.
• Binary class label encoding: Describe how the string labels are converted to integer labels and distinguish these class IDs from GPT vocabulary token IDs.
  Why it matters: The classifier’s two outcomes need numeric labels rather than the original strings.
• Training validation and test splits: Describe the random dataset split, the purpose of each portion, and how the resulting data is saved for reuse.
  Why it matters: Separate saved subsets support training, adjustment, and evaluation without repeating dataset preparation.

Key terms: SMSSpamCollection.tsv, pandas DataFrame, ham, spam, undersample, balanced dataset, integer class labels, token IDs, GPT vocabulary, random_split

6.3 Creating data loaders

This section prepares variable-length spam messages as uniform token sequences and uses PyTorch data loaders to batch them with class labels.

• Padding versus truncation tradeoff: Explain why the spam messages need uniform lengths for batching and why the section pads shorter messages rather than truncating every message to the shortest length.
  Why it matters: The length choice determines whether batching sacrifices message content.
• SpamDataset sequence preparation: Describe how SpamDataset turns CSV text into uniformly sized token-ID inputs, including its padding token and how maximum length is handled across the splits.
  Why it matters: Consistently sized token-ID inputs make the messages usable in batches without exceeding the model’s supported context length.
• Classification batch contents: Describe what the training, validation, and test data loaders return and how the section checks a training batch.
  Why it matters: The model needs each batch’s messages paired with the labels it will learn to predict.

Key terms: truncate, pad, uniformly sized, SpamDataset, GPT-2 tokenizer, <|endoftext|>, 50256, max_length, context length, data loaders

6.4 Initializing a model with pretrained weights

This section loads pretrained GPT weights, checks that the model still generates coherent text, and shows why it needs fine-tuning for spam classification.

• Pretrained GPT initialization: Describe how the section initializes the model for spam-classification fine-tuning using the pretraining configuration and downloaded GPT weights.
  Why it matters: Fine-tuning begins with a pretrained model that can be adapted to the classification task.
• Coherent generation check: Explain why the section generates text after loading the weights and what coherent output indicates.
  Why it matters: This quick check gives a reason to trust the loaded model before adapting it.
• Pretraining versus instruction following: Explain what happens when the pretrained model is prompted to classify spam and why the section considers that result expected.
  Why it matters: It explains why successful weight loading is not the end of preparing a spam classifier.

Key terms: GPTModel, download_and_load_gpt2, load_weights_into_gpt, pretrained weights, text generation utility function, coherent text, model weights, pretraining, instruction fine-tuning, classification fine-tuning

6.5 Adding a classification head

The section adapts a pretrained GPT model for spam classification by replacing its output layer, selecting layers to fine-tune, and using the last token’s output to represent the input text.

• Class-Matched Output Head: Explain how the pretrained model’s vocabulary-sized output layer is replaced with a classification head whose number of outputs matches the number of classes.
  Why it matters: Changing the output head makes the pretrained model produce class-specific outputs instead of vocabulary-wide outputs.
• Selective Layer Fine-Tuning: Describe which layers are frozen and which are made trainable, and explain the reason for fine-tuning selected layers rather than the whole pretrained model.
  Why it matters: Choosing which layers can change lets the model adapt to spam classification without unnecessarily retraining every pretrained layer.
• Last-Token Classification Output: Explain the modified output tensor and why classification fine-tuning focuses on its last token row.
  Why it matters: The last row provides class outputs from the position that can incorporate information from the whole input sequence.

Key terms: out_head, output layer, output nodes, classes, freeze, requires_grad, last transformer block, final LayerNorm, output tensor, last output token

6.6 Calculating the classification loss and accuracy

This section defines how to turn last-token model outputs into spam predictions and how to calculate accuracy and cross-entropy loss for classification fine-tuning.

• Last-token class predictions: Explain how the model predicts spam or non-spam from the last token’s two class outputs, and why softmax is optional when choosing a label.
  Why it matters: This turns the model’s outputs into the labels needed to check its predictions.
• Dataset classification accuracy: Describe how classification accuracy is computed across examples and what the initial accuracies indicate.
  Why it matters: Accuracy shows how often the classifier is right before and after fine-tuning.
• Differentiable classification loss: Justify using cross-entropy loss instead of accuracy for optimization, and describe which model outputs the classification loss uses.
  Why it matters: The loss supplies an optimizable measure aligned with the goal of improving classification accuracy.

Key terms: last token, argmax, softmax, spam, not spam, classification accuracy, calc_accuracy_loader, cross-entropy loss, calc_loss_batch, calc_loss_loader

6.7 Fine-tuning the model on supervised data

The section fine-tunes a pretrained LLM for spam classification, then uses loss and accuracy to judge training progress and generalization.

• Classification training loop changes: Explain how the classification fine-tuning loop differs from the pretraining loop: it tracks examples rather than tokens and calculates accuracy after each epoch instead of generating sample text.
  Why it matters: These changes adapt a familiar training loop to a classification task.
• Loss trends and epoch choice: Describe what the training and validation loss curves show here and how their trends can guide the number of epochs.
  Why it matters: Loss curves help distinguish useful continued training from training that begins to overfit.
• Sampled versus full accuracy: Distinguish the accuracy estimates plotted during training from the later performance calculation across the full training, validation, and test sets.
  Why it matters: A quick training-time estimate should not be mistaken for an evaluation of every example.
• Final accuracy gap interpretation: Explain what the reported training, validation, and test accuracy comparisons suggest about overfitting and why validation accuracy can exceed test accuracy.
  Why it matters: Comparing dataset splits gives a more careful reading of classifier performance than reporting one accuracy alone.

Key terms: train_classifier_simple, examples_seen, classification accuracy, training loss, validation loss, overfitting, epochs, eval_iter, training accuracy, validation accuracy

6.8 Using the LLM as a spam classifier

This section uses a fine-tuned GPT model to classify new messages as spam or not spam, then saves it for later reuse.

• Spam classification workflow: Trace how the fine-tuned GPT-based model classifies new text: preprocess it into token IDs, predict an integer class label, and return the corresponding class name.
  Why it matters: It connects the trained model’s numeric prediction to a label someone can use for a new message.
• Saving for later reuse: Explain why the fine-tuned classifier is saved and name the method the section uses to save it.
  Why it matters: Saving preserves the trained classifier for future use without repeating fine-tuning.

Key terms: classify_review, token IDs, integer class label, class name, spam, not spam, torch.save, save the model, loaded

6.9 Summary

This chapter summary recaps how a pretrained LLM is adapted, trained, and evaluated for text classification.