liveBook · Manning

Post-training

As explained in Machines that Think

Post-training is the adaptation or refinement of a pretrained model for improved behavior, efficiency, or practical deployment.

Post-training adapts a pretrained model for practical use or changes which responses it treats as appropriate. Techniques can refine or compress learned structure; supervised fine-tuning (SFT) uses curated instruction–response examples to make desired patterns more likely without replacing the model’s predictive core. section 4.8 section 7.1.1

Post-training can shape both intended behavior and unintended tendencies:

  • SFT can tune a model for roles such as chatbot, coding helper, support agent, tutor, or domain expert, and can encourage safer responses to sensitive requests. section 7.1.1
  • Patterns encouraged indirectly by a reward signal can spread beyond their original context, especially if they later appear in SFT examples. section 7.1.1
Figure 7.1 Supervised fine-tuning (SFT). The model is trained on curated instruction–response examples that represent desired behavior. During training, the model predicts the response token by token and compares its predictions with the target sequence. Errors adjust the model’s parameters so that responses resembling the reference answers become more likely in future generations.

The figure traces SFT: the model predicts response tokens, compares them with curated targets, and updates parameters to make similar responses more likely. section 7.1.1

How Machines that Think covers it

How deep it goes
Explains SFT’s instruction–response training and behavioral effects, while naming LoRA, distillation, and quantization without detailing them.
Examples
Uses chatbot and specialist roles, safety-oriented responses, and a playful-style tendency that spread across contexts.
What it assumes
Assumes a pretrained or base model and focuses on refinement rather than teaching pretraining from scratch.
What you can do after
Evaluate/understand how curated examples shape model behavior and how unintended tendencies can transfer across contexts.
How it connects
Connects post-training to pretraining, supervised fine-tuning, alignment, and reward signals.

Also explained in

Compare how 5 books explain post-training →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.