Memorization occurs when a model fits training examples without a comparable gain on new examples. A widening gap between falling training loss and nearly unchanged evaluation loss is a warning sign. section 2.8
A 40-example fine-tune
After 20 LoRA steps—about four passes over 40 IT support examples—training loss fell from 2.94 to 1.69, while evaluation loss moved from 1.98 to 1.87. That pattern is expected at this scale: the run demonstrates that the pipeline works, but not that the model learned behavior it can carry to new tickets. section 2.8
Memorization can also expose training data: personally identifiable information should be removed or anonymized unless its use is justified and legally permitted. section 5.11
Listing 2.5 SFTTrainer log over the 20 quickstart steps
{'loss': 2.9427, 'grad_norm': 1.630, 'learning_rate': 0.000178, 'epoch': 1.0}
{'loss': 1.9814, 'grad_norm': 0.453, 'learning_rate': 0.000122, 'epoch': 2.0}
{'eval_loss': 1.9761, 'epoch': 2.0}
{'loss': 1.7807, 'grad_norm': 0.527, 'learning_rate': 0.000067, 'epoch': 3.0}
{'loss': 1.6865, 'grad_norm': 0.427, 'learning_rate': 0.000011, 'epoch': 4.0}
{'eval_loss': 1.8741, 'epoch': 4.0}
{'train_runtime': 78.45, 'train_samples_per_second': 2.04, 'train_loss': 2.0978, 'epoch': 4.0}Compare the training and evaluation losses across the 20 steps. section 2.8





