Rejection sampling
Rejection sampling is a method that generates candidate samples and accepts those meeting a specified criterion while discarding the others.
LLM Evaluation and Alignment, The Foundational Ideas
Rejection sampling is a filtering method that retains generated candidates meeting a specified acceptance criterion and discards the others.
- How it is used
- Filters generated responses into supervised-training data for reward-guided post-training.
- How deep it goes
- Explains candidate generation, reward scoring, thresholding, supervised fine-tuning, iteration, and reward-model failure risks.
- Examples
- Uses iterated post-training rounds and Llama 2’s rejection-sampling stage before final PPO as concrete examples.
- What you can do after
- Apply it via supervised fine-tuning to build filtered training corpora, iterate policy improvement, and inspect reward-model failure modes.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains rejection sampling →
Fabulous Adventures in Data Structures and Algorithms
The rejection algorithm samples candidate outcomes and discards candidates that fail a probabilistic acceptance test.
- How it is used
- Uses easy-to-sample helper distributions to generate categorical or continuous samples that follow desired weights or densities.
- How deep it goes
- Works through dartboard intuition, acceptance code, helper scaling, clamping, and unbounded retry costs.
- Examples
- Demonstrates weighted integer categories, a two-humped continuous density, and a normal distribution clamped to [0.0, 1.0].
- What you can do after
- Build it from scratch in C# using uniform samplers, a helper distribution, a target weight function, and an acceptance loop.
Read how Fabulous Adventures in Data Structures and Algorithms explains rejection sampling →
Build a DeepSeek Model (From Scratch)
Rejection sampling is a data curation procedure that generates multiple candidates and retains only those meeting specified correctness and quality criteria.
- How it is used
- Curates high-quality supervised training data from multiple model-generated candidates during post-training and distillation.
- How deep it goes
- Explains generation, verification, quality filtering, and dataset integration, without detailing a software implementation.
- Examples
- Uses ground-truth math checks, executable code tests, model judgments, formatting filters, and an 800K-sample distillation dataset.
- What you can do after
- Build it from scratch by generating candidates, applying correctness and quality filters, and assembling accepted responses into supervised data.
Read how Build a DeepSeek Model (From Scratch) explains rejection sampling →
Reinforcement Learning from Human Feedback
Rejection sampling is a sampling method that accepts or filters candidate outputs according to a specified criterion or scoring procedure.
- How it is used
- Deploys reward-model filtering as a simple alternative to reinforcement learning for preference fine-tuning.
- How deep it goes
- Explains four pipeline stages and selection strategies, but defers prompt, reward-model, sequencing, and deduplication choices to experimentation.
- Examples
- Uses WebGPT, Helpful and Harmless, process reward models, Llama 2 Chat, and DeepSeek R1 as pipeline examples.
- What you can do after
- Operate/configure a generate-rank-fine-tune pipeline and choose between per-prompt and global completion selection.
Read how Reinforcement Learning from Human Feedback explains rejection sampling →
Building LLM Applications with DSPy
Rejection sampling is a method that accepts candidate samples meeting a criterion and rejects those that do not.
- How it is used
- Uses rejection sampling during DSPy bootstrapping to select safe training examples as demonstrations.
- How deep it goes
- Describes metric-based acceptance, training-set order, and repeated rounds without formalizing the sampling method.
- Examples
- Uses exact-match intent labels to accept correct predictions and reject incorrect ones.
- What you can do after
- Apply rejection sampling with DSPy's BootstrapFewShot to compile a program using accepted training examples as demonstrations.
Read how Building LLM Applications with DSPy explains rejection sampling →
Also covered in
What to know first
- chain-of-thought promptingChain-of-thought prompting is a technique that elicits or supplies intermediate reasoning steps to guide a model toward a final answer.
- DeepSeek-R1-ZeroDeepSeek-R1-Zero is a reasoning language model trained with reinforcement learning without supervised fine-tuning as an initial warm-start stage.
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- policyA policy is a rule, principle, or decision function that governs behavior or determines actions or permissions under specified conditions.
Related concepts
- data curationkind of · Data curation is the systematic selection, review, cleaning, organization, and maintenance of data for a particular purpose.
- reward modeluses · A reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
- reinforcement learningalternative to · Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.
Questions about rejection sampling
What is rejection sampling?
Rejection sampling is a method that generates candidate samples and accepts those meeting a specified criterion while discarding the others.
Which Manning books explain rejection sampling?
LLM Evaluation and Alignment, The Foundational Ideas, Fabulous Adventures in Data Structures and Algorithms, Build a DeepSeek Model (From Scratch), Reinforcement Learning from Human Feedback and Building LLM Applications with DSPy.
What should I know before learning rejection sampling?
Manning books that teach rejection sampling usually introduce chain-of-thought prompting, DeepSeek-R1-Zero, fine-tuning and policy first.
What is the difference between rejection sampling and reinforcement learning?
Rejection sampling is a method that generates candidate samples and accepts those meeting a specified criterion while discarding the others. Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.
What is rejection sampling a kind of?
Rejection sampling is a kind of data curation. Data curation is the systematic selection, review, cleaning, organization, and maintenance of data for a particular purpose.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




