chapter six

6 Supervised fine-tuning: Maximum expressiveness

 

This chapter covers

  • Training a complete Qwen3-4B model on the IT support dataset using TRL’s SFTTrainer
  • Monitoring loss curves and gradient norms to catch problems early
  • Evaluating base versus fine-tuned performance with quantitative metrics and behavioral tests
  • Deciding when full SFT outperforms LoRA and when it is overkill
  • Detecting safety regression after training

In chapter 5, you trained a LoRA adapter that improved your model’s instruction-following ability while updating less than 1% of its parameters. For many tasks, that is enough. But suppose your LoRA-tuned IT support model still produces inconsistent output formats, sometimes returning a numbered list, sometimes a paragraph, sometimes a half-finished sentence. You have tried increasing LoRA rank from 8 to 32, and the improvement has plateaued. The low-rank constraint is limiting how thoroughly the model can reshape its behavior.

6.1 Hands-on: SFT training workflow

6.1.1 Data preparation

6.1.2 Model loading and training configuration

6.1.3 Training script

6.1.4 Inference

6.2 Monitoring training

6.2.1 Tools for monitoring training

6.2.2 Reading the loss curves

6.2.3 Red flags and concrete thresholds

6.3 Evaluating the fine-tuned model

6.3.1 Evaluation strategy

6.3.2 Hands-on evaluation code

6.3.3 Making decisions from evaluation results

6.4 When SFT delivers value

6.4.1 Realistic expectations

6.5 Fine-tuning refines, doesn’t inject knowledge

6.6 SFT vs. LoRA: Realistic performance expectations

6.6.1 The cost-performance tradeoff

6.6.2 The forgetting risk (catastrophic forgetting)

6.7 Data requirements

6.7.1 Quality over quantity

6.8 Common pitfalls and how to avoid them

6.8.1 Avoiding catastrophic forgetting

6.8.2 Overfitting

6.8.3 Distribution mismatch

6.12.3 Results