Post-training adapts a pretrained model toward useful reasoning and behavior through additional optimization, generated data, supervised examples, and alignment procedures. section 7.0 section 1.2.3
A practical post-training pipeline can proceed through five increasingly capable stages:
- Start with a lightly fine-tuned base model and a relatively small cold-start dataset. section 1.2.3
- Apply pure reinforcement learning so the model explores reasoning patterns through trial and error. section 1.2.3 section 7.0
- Use rejection sampling to select strong responses and create synthetic training data. section 1.2.3
- Blend synthetic data with supervised examples to balance response quality and domain breadth. section 1.2.3
- Finish with reinforcement learning across diverse prompts to improve robustness and generalization. section 1.2.3
DeepSeek-R1
DeepSeek-R1 illustrates the pipeline: DeepSeek-V3 served as the base, reinforcement learning encouraged reasoning, self-generated responses supplied training data, and a final RL phase improved performance across task types. Knowledge distillation then transferred capabilities from a large teacher into smaller student checkpoints, including 1.5B, 7B, 8B, 14B, 32B, and 70B models. section 1.2.3
Figure 1.6 The multi-step post-training pipeline used to create DeepSeek-R1 from the DeepSeek-V3 base model. This process involves a combination of reinforcement learning (Pure RL), data generation (Rejection sampling), and fine-tuning to instill advanced reasoning capabilities.

Use the pipeline diagram to trace how fine-tuning, pure reinforcement learning, rejection sampling, data blending, and final RL combine into a reasoning-oriented model. section 1.2.3




