The DeepSeek distillation dataset contains approximately 800,000 teacher-generated samples: about 600,000 reasoning traces and 200,000 non-reasoning examples. It supports supervised fine-tuning of dense student models rather than classical temperature-scaled KL distillation. section 8.5.2 section 8.9
Build the dataset through rejection sampling:
- Generate multiple candidate responses for each prompt with the DeepSeek-R1 teacher. section 8.5.2
- Verify mathematical answers against ground truth and execute code responses to test their outputs. section 8.5.2
- Use DeepSeek-V3 as a generative reward model for STEM reasoning and logical puzzles without one definitive answer. section 8.5.2
- Discard responses that are correct but chaotic, mixed-language, excessively code-heavy, or incoherently formatted. section 8.5.2
Estimated composition and results
The estimated domain split is 395,000 math samples, 211,000 code, 10,000 STEM, 10,000 logic, and 178,000 general samples. A distilled 32B model reaches 72.6% on AIME, compared with 47.0% for a reinforcement-learning-only counterpart; a 1.5B student reaches 83.9% on MATH but 16.9% on LiveCodeBench. section 8.9
Figure 8.16 The rejection sampling process for curating DeepSeek-R1’s distillation dataset. Multiple responses are generated per prompt, verified for correctness, filtered for quality, and only clean, correct answers are kept.

Trace the curation pipeline from multiple teacher responses through correctness verification and formatting filters to retained samples. section 8.5.2
Figure 8.17 DeepSeek-R1’s complete training pipeline. Distillation (Stage 6) is the final stage, where open-source dense models are fine-tuned on 800K samples curated from the teacher via rejection sampling. Data generation costs only $10K versus $202K for RL training. (Cost figures are estimates derived from the H800-hour counts disclosed in the DeepSeek-R1 paper at published cloud GPU rates; exact internal costs may differ.)

Locate distillation as Stage 6, where open-source dense models are fine-tuned on the curated dataset. section 8.5.2
Figure 8.18 The composition of DeepSeek-R1’s 800K distillation dataset by domain. Math dominates at 395K samples (49%), followed by Code at 211K (26%), with General, STEM, and Logic making up the remainder. (Estimated split; the R1 paper itself reports only the ~600K reasoning / ~200K non-reasoning totals.)

Inspect the estimated distribution across math, code, STEM, logic, and general domains. section 8.5.2
