liveBook · Manning

Overfitting

As explained in LLM Customization and Fine-Tuning

A model-learning failure in which performance worsens on inputs that differ from the training distribution because the model has fit the training data too narrowly.

A model can achieve low training loss yet fail on new inputs because it has memorized examples rather than learned transferable patterns. This risk is especially acute in supervised fine-tuning (SFT) with small or narrowly varied datasets. section 6.8.2 section 6.7.1

Overfitting also affects prompt design: repeated iteration can select a prompt that fits the design examples but performs worse on production inputs. Keep evaluation data separate from training data, and test on genuinely unseen examples rather than repeatedly tuning to the same evaluation set. section 4.4 section 5.10

Read the loss curves

If training loss keeps falling while validation loss rises, stop at the checkpoint with the lowest validation loss. A training loss of 0.3 and validation loss of 0.9 illustrates a large gap; for datasets of 400 to 1,000 examples, a gap above 0.3 is a strong indicator of overfitting. section 6.2.2 section 6.8.2

Reduce the risk during fine-tuning:

  • Use early stopping based on validation loss and restore the best checkpoint rather than assuming the last epoch is best. section 6.8.2
  • Add regularization: start with weight decay of 0.01; for severe overfitting, consider 0.05 or attention-layer dropout. section 6.8.2
  • Improve data diversity across question types, edge cases, entity types, and difficulty levels. Increasing volume helps most when the additional examples broaden coverage. section 6.7.1 section 6.8.2
  • If full SFT still overfits, consider LoRA's lower trainable capacity. For fewer than 500 examples, it often generalizes better than full SFT. section 6.8.2
from transformers import EarlyStoppingCallback
 
trainer = SFTTrainer(
    ...
    callbacks=[EarlyStoppingCallback(early_stopping_patience=2)],
)

Configure a Trainer callback to stop after two evaluations without improvement. section 6.8.2

Test separately for behavioral regression: strong target-task metrics do not establish that general capabilities and safety behavior remain intact. A 15% general-purpose instruction mix is a starting point for preserving breadth alongside domain specialization. Reevaluate with current queries as production inputs drift. section 6.14.4 section 5.10

How LLM Customization and Fine-Tuning covers it

How deep it goes
Explains loss-curve diagnosis and gives early-stopping, regularization, data-diversity, and capacity-reduction remedies with practical starting values.
Examples
Uses a 0.3 training versus 0.9 validation loss example and a prompt scoring 89% on its design examples.
What you can do after
Diagnoses overfitting with loss curves and mitigates it through early stopping, regularization, more diverse data, or reduced model capacity.
How it connects
Connects overfitting to validation loss, early stopping, regularization, LoRA, behavioral regression, and distribution shift.

Also explained in

Compare how 33 books explain overfitting →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.