Post-training
Post-training is the additional training and adaptation of a pretrained model to improve its capabilities, behavior, alignment, or usefulness for particular tasks.
Reinforcement Learning from Human Feedback
Post-training is the set of optimization and adaptation procedures applied to a pretrained model to improve its usefulness for downstream tasks or alignment objectives.
- How it is used
- Positions post-training as the process for turning pretrained language-model capabilities into useful downstream behavior.
- How deep it goes
- Explains a three-stage RLHF pipeline, alternative optimization tools, and multistage recipes with data scales and practical trade-offs.
- Examples
- Illustrates a three-stage InstructGPT recipe and a Tülu 3 recipe using instruction data, preference pairs, and verifiable-reward prompts.
- What you can do after
- Evaluate and understand post-training recipes by comparing their stages, data choices, and potential optimization trade-offs.
Read how Reinforcement Learning from Human Feedback explains post-training →
Introduction to Generative AI, Second Edition
Post-training is the adaptation of a pretrained model using additional task-specific data and optimization rather than training the model from scratch.
- How it is used
- Uses post-training to adapt pretrained language models for specific abilities and safer responses.
- How deep it goes
- Explains weight adaptation and compares training stages and prompting, while outlining safety methods without implementation details.
- Examples
- Uses instruction following, code generation, chatbot concision, and safe responses to unsafe inputs as examples.
- What you can do after
- Evaluate and understand when post-training may be preferable to prompting and what data, skills, and safety tradeoffs it entails.
Read how Introduction to Generative AI, Second Edition explains post-training →
Build a Reasoning Model (From Scratch)
Methods applied after initial pretraining to improve model behavior or capabilities.
- How it is used
- Positions model refinement after pretraining to improve instruction following, preference alignment, and reasoning behavior.
- How deep it goes
- Distinguishes major stages and their ordering but defers detailed reinforcement-learning implementation to later treatment.
- Examples
- Compares pretrained, instruction-tuned, and preference-tuned answers to a sleep-and-health summary prompt.
- What you can do after
- Evaluate how training stages change instruction following, response style, and reasoning-like behavior.
Read how Build a Reasoning Model (From Scratch) explains post-training →
Build a DeepSeek Model (From Scratch)
Post-training is the stage of adapting a pretrained model through additional optimization, data generation, or alignment procedures.
- How it is used
- Deploys additional optimization and data-generation stages to turn a capable base model into a more useful reasoning model.
- How deep it goes
- Explains a five-stage pipeline and distillation, but stops short of proprietary data, exact weights, and massive distributed infrastructure.
- Examples
- Uses DeepSeek-R1, DeepSeek-V3, verifiable rewards, synthetic responses, and distilled checkpoints ranging from 1.5B to 70B parameters.
- What you can do after
- Build it from scratch by implementing reinforcement-learning and distillation techniques for adapting a pretrained language model.
Read how Build a DeepSeek Model (From Scratch) explains post-training →
Machines that Think
Post-training is the adaptation or refinement of a pretrained model for improved behavior, efficiency, or practical deployment.
- How it is used
- Positions post-training as a way to adapt or compress pretrained models for practical use and shape their responses for specific roles.
- How deep it goes
- Explains SFT’s instruction–response training and behavioral effects, while naming LoRA, distillation, and quantization without detailing them.
- Examples
- Uses chatbot and specialist roles, safety-oriented responses, and a playful-style tendency that spread across contexts.
- What you can do after
- Evaluate/understand how curated examples shape model behavior and how unintended tendencies can transfer across contexts.
Also covered in
- LLM Evaluation and Alignment, The Foundational IdeasIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- AI agentAn AI agent is a computational system that uses artificial intelligence to interpret information and select or perform actions toward goals while interacting with an environment.
- chain-of-thought promptingChain-of-thought prompting is a technique that elicits or supplies intermediate reasoning steps to guide a model toward a final answer.
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
Related concepts
- fine-tuninghas part · Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- foundation modeluses · A foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks.
- reinforcement learninghas part · Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.
- Direct Preference Optimizationincludes · Direct Preference Optimization is a technique for training a model directly from preference data, such as preferred and dispreferred outputs, without separately fitting a reward model.
- distillationhas part · Distillation is a training technique in which a student model learns to approximate the outputs or behavior of a teacher model.
- personality trainingincludes · Personality training is model training that shapes recurring stylistic, behavioral, or character-related characteristics in generated responses.
Questions about post-training
What is post-training?
Post-training is the additional training and adaptation of a pretrained model to improve its capabilities, behavior, alignment, or usefulness for particular tasks.
Which Manning books explain post-training?
Reinforcement Learning from Human Feedback, Introduction to Generative AI, Second Edition, Build a Reasoning Model (From Scratch), Build a DeepSeek Model (From Scratch) and Machines that Think.
What should I know before learning post-training?
Manning books that teach post-training usually introduce language model, AI agent, chain-of-thought prompting and fine-tuning first.
What kinds of post-training are there?
Kinds of post-training include Direct Preference Optimization and personality training.
What are the parts of post-training?
Post-training includes fine-tuning, reinforcement learning and distillation.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




