liveBook · Manning

Supervised fine-tuning

Also called SFT

Supervised fine-tuning is further training a pretrained model on labeled input-output examples to adapt its behavior to desired outputs.

How 3 Manning books explain it

LLM Customization and Fine-Tuning

Further training a pretrained model on labeled input-output examples to adapt its behavior.

How it is used
Adapts pretrained models to domain behavior and task structure when cheaper interventions fall short.
How deep it goes
Works through data preparation, full-parameter training, monitoring, evaluation, safety checks, and the decision to escalate from LoRA.
Examples
Trains a Qwen3-4B IT support assistant and compares its held-out Token-F1 with base and LoRA models.
What you can do after
Apply full SFT via TRL’s SFTTrainer, evaluate the resulting model, and assess whether its gains justify the extra cost.

Read how LLM Customization and Fine-Tuning explains supervised fine-tuning →

Applied Reinforcement Learning

Supervised fine-tuning is the training of a pre-trained model on labeled examples so that it produces desired outputs for given inputs.

How it is used
Aligns an existing language model with demonstrated domain-specific behavior before preference modeling and reinforcement learning.
How deep it goes
Details response-only label masking, dataset preparation, and training choices, while omitting standard evaluation and checkpointing boilerplate.
Examples
Uses medical chatbot demonstrations to teach empathy, appropriate safety language, and emergency escalation.
What you can do after
Build an SFT training stage that learns from curated prompt-response pairs and computes loss only on response tokens.

Read how Applied Reinforcement Learning explains supervised fine-tuning →

Machines that Think

Supervised fine-tuning adjusts a pretrained model using labeled examples of desired inputs and outputs or behaviors.

How it is used
Shapes pretrained language models for application roles and introduces desired or safer response patterns.
How deep it goes
Explains instruction–response training through token prediction, target comparison, and parameter updates, then distinguishes preference-based alignment.
Examples
Uses chatbot, coding helper, customer-support agent, and tutor roles, along with refusals and intermediate-step problem solving, to illustrate desired behavior.
What you can do after
Evaluate or understand how curated examples change response tendencies without guaranteeing ethical behavior.

Read how Machines that Think explains supervised fine-tuning →

What to know first

Related concepts

Questions about supervised fine-tuning

What is supervised fine-tuning?

Supervised fine-tuning is further training a pretrained model on labeled input-output examples to adapt its behavior to desired outputs.

Which Manning books explain supervised fine-tuning?

LLM Customization and Fine-Tuning, Applied Reinforcement Learning and Machines that Think.

What should I know before learning supervised fine-tuning?

Manning books that teach supervised fine-tuning usually introduce objective function, reward model, Proximal Policy Optimization and weight decay first.

What is the difference between supervised fine-tuning and low-rank adaptation?

Supervised fine-tuning is further training a pretrained model on labeled input-output examples to adapt its behavior to desired outputs. Low-rank adaptation (LoRA) is a parameter-efficient fine-tuning technique that adapts a model by learning low-rank updates to its weights, typically while keeping most original weights fixed.

What kinds of supervised fine-tuning are there?

Kinds of supervised fine-tuning include distillation.

What is supervised fine-tuning also called?

Supervised fine-tuning is also called SFT.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.