Post-training adapts a pretrained model for practical use or changes which responses it treats as appropriate. Techniques can refine or compress learned structure; supervised fine-tuning (SFT) uses curated instruction–response examples to make desired patterns more likely without replacing the model’s predictive core. section 4.8 section 7.1.1
Post-training can shape both intended behavior and unintended tendencies:
- SFT can tune a model for roles such as chatbot, coding helper, support agent, tutor, or domain expert, and can encourage safer responses to sensitive requests. section 7.1.1
- Patterns encouraged indirectly by a reward signal can spread beyond their original context, especially if they later appear in SFT examples. section 7.1.1
Figure 7.1 Supervised fine-tuning (SFT). The model is trained on curated instruction–response examples that represent desired behavior. During training, the model predicts the response token by token and compares its predictions with the target sequence. Errors adjust the model’s parameters so that responses resembling the reference answers become more likely in future generations.

The figure traces SFT: the model predicts response tokens, compares them with curated targets, and updates parameters to make similar responses more likely. section 7.1.1




