liveBook · Manning

Post-training

Post-training is the additional training and adaptation of a pretrained model to improve its capabilities, behavior, alignment, or usefulness for particular tasks.

How 5 Manning books explain it

Reinforcement Learning from Human Feedback

Post-training is the set of optimization and adaptation procedures applied to a pretrained model to improve its usefulness for downstream tasks or alignment objectives.

How it is used
Positions post-training as the process for turning pretrained language-model capabilities into useful downstream behavior.
How deep it goes
Explains a three-stage RLHF pipeline, alternative optimization tools, and multistage recipes with data scales and practical trade-offs.
Examples
Illustrates a three-stage InstructGPT recipe and a Tülu 3 recipe using instruction data, preference pairs, and verifiable-reward prompts.
What you can do after
Evaluate and understand post-training recipes by comparing their stages, data choices, and potential optimization trade-offs.

Read how Reinforcement Learning from Human Feedback explains post-training →

Introduction to Generative AI, Second Edition

Post-training is the adaptation of a pretrained model using additional task-specific data and optimization rather than training the model from scratch.

How it is used
Uses post-training to adapt pretrained language models for specific abilities and safer responses.
How deep it goes
Explains weight adaptation and compares training stages and prompting, while outlining safety methods without implementation details.
Examples
Uses instruction following, code generation, chatbot concision, and safe responses to unsafe inputs as examples.
What you can do after
Evaluate and understand when post-training may be preferable to prompting and what data, skills, and safety tradeoffs it entails.

Read how Introduction to Generative AI, Second Edition explains post-training →

Build a Reasoning Model (From Scratch)

Methods applied after initial pretraining to improve model behavior or capabilities.

How it is used
Positions model refinement after pretraining to improve instruction following, preference alignment, and reasoning behavior.
How deep it goes
Distinguishes major stages and their ordering but defers detailed reinforcement-learning implementation to later treatment.
Examples
Compares pretrained, instruction-tuned, and preference-tuned answers to a sleep-and-health summary prompt.
What you can do after
Evaluate how training stages change instruction following, response style, and reasoning-like behavior.

Read how Build a Reasoning Model (From Scratch) explains post-training →

Build a DeepSeek Model (From Scratch)

Post-training is the stage of adapting a pretrained model through additional optimization, data generation, or alignment procedures.

How it is used
Deploys additional optimization and data-generation stages to turn a capable base model into a more useful reasoning model.
How deep it goes
Explains a five-stage pipeline and distillation, but stops short of proprietary data, exact weights, and massive distributed infrastructure.
Examples
Uses DeepSeek-R1, DeepSeek-V3, verifiable rewards, synthetic responses, and distilled checkpoints ranging from 1.5B to 70B parameters.
What you can do after
Build it from scratch by implementing reinforcement-learning and distillation techniques for adapting a pretrained language model.

Read how Build a DeepSeek Model (From Scratch) explains post-training →

Machines that Think

Post-training is the adaptation or refinement of a pretrained model for improved behavior, efficiency, or practical deployment.

How it is used
Positions post-training as a way to adapt or compress pretrained models for practical use and shape their responses for specific roles.
How deep it goes
Explains SFT’s instruction–response training and behavioral effects, while naming LoRA, distillation, and quantization without detailing them.
Examples
Uses chatbot and specialist roles, safety-oriented responses, and a playful-style tendency that spread across contexts.
What you can do after
Evaluate/understand how curated examples shape model behavior and how unintended tendencies can transfer across contexts.

Read how Machines that Think explains post-training →

Also covered in

What to know first

Related concepts

Questions about post-training

What is post-training?

Post-training is the additional training and adaptation of a pretrained model to improve its capabilities, behavior, alignment, or usefulness for particular tasks.

Which Manning books explain post-training?

Reinforcement Learning from Human Feedback, Introduction to Generative AI, Second Edition, Build a Reasoning Model (From Scratch), Build a DeepSeek Model (From Scratch) and Machines that Think.

What should I know before learning post-training?

Manning books that teach post-training usually introduce language model, AI agent, chain-of-thought prompting and fine-tuning first.

What kinds of post-training are there?

Kinds of post-training include Direct Preference Optimization and personality training.

What are the parts of post-training?

Post-training includes fine-tuning, reinforcement learning and distillation.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.