liveBook · Manning

Supervised fine-tuning

As explained in LLM Customization and Fine-Tuning

Further training a pretrained model on labeled input-output examples to adapt its behavior.

Supervised fine-tuning (SFT) adapts a pretrained model using labeled input–output examples. In full SFT, every model parameter is updated, making task behavior, domain vocabulary, and response patterns part of the model’s defaults. The extra capacity comes with higher memory, training, and monitoring costs than LoRA, which trains small adapters on a frozen base model. section 6.0 section 6.1

Start with the least expensive intervention that can meet the quality target. Prompting or a decode-time schema may be enough for format alone; retrieval-augmented generation (RAG) is better suited to frequently changing knowledge or verifiable citations. Start training with LoRA, then consider full SFT when a well-tuned adapter plateaus on a meaningful quality gap. section 1.6.2 section 1.7 section 5.2.3

Before committing to full SFT, check the decision against production constraints:

  • Use a shared held-out evaluation set to compare the base model, prompting, RAG, LoRA, and SFT where applicable. Resolving differences of a few points typically requires 200–500 evaluation examples. section 1.7
  • Treat 1,000 curated examples as a floor for testing full SFT, not a guarantee. Plan for it at 10,000 or more; at 100,000 or more, LoRA capacity may become a bottleneck. section 5.2.3 section 6.13
  • Budget for GPUs, training time, safety checks, and ongoing maintenance. Full SFT often offers only single-digit percentage improvements over well-tuned LoRA, so measure whether the gain has business value. section 6.4.1 section 6.13

Curate representative, diverse examples with correct, complete assistant responses and consistent conventions. Strip sensitive information and normalize noisy records before converting them to chat-structured JSONL. In the illustrated setup, system and user turns condition the model, while loss is computed only on assistant tokens; inconsistencies in those targets become training signals. section 6.7 section 6.7.1

A practical full-SFT workflow is:

  • Prepare domain-filtered training data and separate validation and test examples. Establish a baseline and preserve the test set for final comparisons. section 6.1 section 8.2.7
  • Load the pretrained model and train without a LoRA configuration. For 4B–7B models, starting settings include a 2e-5 learning rate, small per-device batches, gradient accumulation, and gradient checkpointing. section 6.1.3 section 6.1.2
  • Run a smoke test, monitor training and validation loss and gradient norms, and select a suitable checkpoint. Rising validation loss can indicate overfitting; sudden gradient spikes can signal divergence. section 6.1.3 section 6.8
  • Inspect generated answers, compare task metrics on held-out examples, and run safety regression checks. Also test general capabilities: aggressive updates can cause catastrophic forgetting. section 6.1.4 section 6.8
Table 6.1 Starting hyperparameters for SFT on 4B to 7B models
Parameter
Recommended
Notes
num_train_epochs
3
Rarely need more than 5; watch val loss
per_device_train_batch_size
1 to 2
Limited by GPU VRAM
gradient_accumulation_steps
8 to 16
An effective batch of 16 is a good default
learning_rate
2e-5
Range: 1e-5 to 5e-5
warmup_ratio
0.03 to 0.10
Or 50 to 200 fixed steps
weight_decay
0.01
Mild regularization
max_length
512 to 1024
Must cover your longest examples
bf16
True
Use fp16 only if bf16 unsupported
gradient_checkpointing
True
Essential for consumer GPUs
save_total_limit
2 to 3
Disk space: intermediate checkpoints are about 24 GB each with optimizer state; the final model is about 8 GB

Starting hyperparameters show the memory and stability choices for full SFT on 4B–7B models. section 6.1.2

IT support assistant

A Qwen3-4B assistant is trained on 450 IT support examples to produce step-by-step answers. On a 50-question held-out test, Token-F1 was 0.153 for the base model, 0.154 for LoRA, and 0.168 for full SFT. The test is small: differences of a few points are within its noise, so this run demonstrates the workflow rather than proving a reliable SFT advantage. The 450-example run is illustrative; the decision floor for testing full SFT remains about 1,000 curated examples. section 6.1.3 section 5.2.3 section 8.2.7 section 6.13

SFT learns from target responses, not explicit comparisons between acceptable answers. If the model can already perform the task but needs subtler preferences for concision, tone, or safety, preference optimization can refine an SFT checkpoint; it is not needed when SFT already meets the quality bar. section 6.10 section 6.10.1

How LLM Customization and Fine-Tuning covers it

How deep it goes
Works through data preparation, full-parameter training, monitoring, evaluation, safety checks, and the decision to escalate from LoRA.
Examples
Trains a Qwen3-4B IT support assistant and compares its held-out Token-F1 with base and LoRA models.
What it assumes
Builds on a LoRA workflow and assumes access to curated task examples and a held-out evaluation set.
What you can do after
Apply full SFT via TRL’s SFTTrainer, evaluate the resulting model, and assess whether its gains justify the extra cost.
How it connects
Compares prompting, retrieval-augmented generation, and LoRA with full SFT, then connects SFT to preference optimization.

Also explained in

Compare how 3 books explain supervised fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.