Supervised fine-tuning
Supervised fine-tuning is further training a pretrained model on labeled input-output examples to adapt its behavior to desired outputs.
LLM Customization and Fine-Tuning
Further training a pretrained model on labeled input-output examples to adapt its behavior.
- How it is used
- Adapts pretrained models to domain behavior and task structure when cheaper interventions fall short.
- How deep it goes
- Works through data preparation, full-parameter training, monitoring, evaluation, safety checks, and the decision to escalate from LoRA.
- Examples
- Trains a Qwen3-4B IT support assistant and compares its held-out Token-F1 with base and LoRA models.
- What you can do after
- Apply full SFT via TRL’s SFTTrainer, evaluate the resulting model, and assess whether its gains justify the extra cost.
Read how LLM Customization and Fine-Tuning explains supervised fine-tuning →
Applied Reinforcement Learning
Supervised fine-tuning is the training of a pre-trained model on labeled examples so that it produces desired outputs for given inputs.
- How it is used
- Aligns an existing language model with demonstrated domain-specific behavior before preference modeling and reinforcement learning.
- How deep it goes
- Details response-only label masking, dataset preparation, and training choices, while omitting standard evaluation and checkpointing boilerplate.
- Examples
- Uses medical chatbot demonstrations to teach empathy, appropriate safety language, and emergency escalation.
- What you can do after
- Build an SFT training stage that learns from curated prompt-response pairs and computes loss only on response tokens.
Read how Applied Reinforcement Learning explains supervised fine-tuning →
Machines that Think
Supervised fine-tuning adjusts a pretrained model using labeled examples of desired inputs and outputs or behaviors.
- How it is used
- Shapes pretrained language models for application roles and introduces desired or safer response patterns.
- How deep it goes
- Explains instruction–response training through token prediction, target comparison, and parameter updates, then distinguishes preference-based alignment.
- Examples
- Uses chatbot, coding helper, customer-support agent, and tutor roles, along with refusals and intermediate-step problem solving, to illustrate desired behavior.
- What you can do after
- Evaluate or understand how curated examples change response tendencies without guaranteeing ethical behavior.
Read how Machines that Think explains supervised fine-tuning →
What to know first
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- reward modelA reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
- Proximal Policy OptimizationProximal Policy Optimization is a policy-gradient reinforcement-learning algorithm that limits policy updates to keep successive policies close and promote stable learning.
- weight decayA regularization technique that reduces model parameter magnitudes during optimization, either by adding a penalty to the objective or by directly shrinking parameters.
Related concepts
- distillationincludes · Distillation is a training technique in which a student model learns to approximate the outputs or behavior of a teacher model.
- low-rank adaptationcontrasts with · Low-rank adaptation (LoRA) is a parameter-efficient fine-tuning technique that adapts a model by learning low-rank updates to its weights, typically while keeping most original weights fixed.
- preference tuningcontrasts with · Preference tuning is a model-training technique that adjusts a model’s behavior toward outputs favored by preference signals, judgments, objectives, or rewards.
- RLHFcontrasts with · Reinforcement learning from human feedback is a model-training approach that uses human evaluations or preferences as feedback to optimize model behavior.
- retrieval-augmented generationcontrasts with · Retrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
- SFTConfigconfigured by · SFTConfig is a configuration class for specifying settings used in supervised fine-tuning of a model.
Questions about supervised fine-tuning
What is supervised fine-tuning?
Supervised fine-tuning is further training a pretrained model on labeled input-output examples to adapt its behavior to desired outputs.
Which Manning books explain supervised fine-tuning?
LLM Customization and Fine-Tuning, Applied Reinforcement Learning and Machines that Think.
What should I know before learning supervised fine-tuning?
Manning books that teach supervised fine-tuning usually introduce objective function, reward model, Proximal Policy Optimization and weight decay first.
What is the difference between supervised fine-tuning and low-rank adaptation?
Supervised fine-tuning is further training a pretrained model on labeled input-output examples to adapt its behavior to desired outputs. Low-rank adaptation (LoRA) is a parameter-efficient fine-tuning technique that adapts a model by learning low-rank updates to its weights, typically while keeping most original weights fixed.
What kinds of supervised fine-tuning are there?
Kinds of supervised fine-tuning include distillation.
What is supervised fine-tuning also called?
Supervised fine-tuning is also called SFT.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


