Rejection sampling is a data-curation procedure: generate multiple responses for each prompt, verify them, and retain only candidates that meet correctness and quality criteria. The accepted responses become supervised training data rather than model weights copied from the generator. section 7.7.3 section 8.5.2
A practical rejection-sampling pipeline proceeds as follows:
- Generate several candidate responses from a capable model, often one strengthened through reasoning-oriented reinforcement learning. section 7.7.1
- Check correctness with ground-truth comparisons, rule-based verifiers, code execution, or model-based judgments when no single answer exists. section 8.5.2
- Filter technically correct candidates for readability, language consistency, coherent formatting, and instruction-following quality. section 7.7.1 section 8.5.2
- Mix accepted reasoning traces with broader supervised examples before fine-tuning, restoring capabilities that verifier-based RL does not directly optimize. section 7.7.1
Math and code curation
For a math prompt, compare each candidate's final answer with the ground truth. For a programming prompt, execute the candidate and test its output. Reject responses that pass correctness checks but contain mixed languages, excessive code blocks, or incoherent formatting; retain only clean, correct traces for training. section 8.5.2
Figure 8.16 The rejection sampling process for curating DeepSeek-R1’s distillation dataset. Multiple responses are generated per prompt, verified for correctness, filtered for quality, and only clean, correct answers are kept.

Use the pipeline diagram to trace candidate generation, correctness verification, quality filtering, and retention of clean responses for distillation. section 8.5.2
Rejection sampling is not reinforcement learning itself. RL improves the generator; rejection sampling harvests its strongest outputs and converts them into a filtered supervised dataset. DeepSeek-R1 used this approach for an approximately 800K-sample distillation dataset. section 7.7.3 section 8.9




