liveBook · Manning

Rejection sampling

Rejection sampling is a method that generates candidate samples and accepts those meeting a specified criterion while discarding the others.

How 5 Manning books explain it

LLM Evaluation and Alignment, The Foundational Ideas

Rejection sampling is a filtering method that retains generated candidates meeting a specified acceptance criterion and discards the others.

How it is used
Filters generated responses into supervised-training data for reward-guided post-training.
How deep it goes
Explains candidate generation, reward scoring, thresholding, supervised fine-tuning, iteration, and reward-model failure risks.
Examples
Uses iterated post-training rounds and Llama 2’s rejection-sampling stage before final PPO as concrete examples.
What you can do after
Apply it via supervised fine-tuning to build filtered training corpora, iterate policy improvement, and inspect reward-model failure modes.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains rejection sampling →

Fabulous Adventures in Data Structures and Algorithms

The rejection algorithm samples candidate outcomes and discards candidates that fail a probabilistic acceptance test.

How it is used
Uses easy-to-sample helper distributions to generate categorical or continuous samples that follow desired weights or densities.
How deep it goes
Works through dartboard intuition, acceptance code, helper scaling, clamping, and unbounded retry costs.
Examples
Demonstrates weighted integer categories, a two-humped continuous density, and a normal distribution clamped to [0.0, 1.0].
What you can do after
Build it from scratch in C# using uniform samplers, a helper distribution, a target weight function, and an acceptance loop.

Read how Fabulous Adventures in Data Structures and Algorithms explains rejection sampling →

Build a DeepSeek Model (From Scratch)

Rejection sampling is a data curation procedure that generates multiple candidates and retains only those meeting specified correctness and quality criteria.

How it is used
Curates high-quality supervised training data from multiple model-generated candidates during post-training and distillation.
How deep it goes
Explains generation, verification, quality filtering, and dataset integration, without detailing a software implementation.
Examples
Uses ground-truth math checks, executable code tests, model judgments, formatting filters, and an 800K-sample distillation dataset.
What you can do after
Build it from scratch by generating candidates, applying correctness and quality filters, and assembling accepted responses into supervised data.

Read how Build a DeepSeek Model (From Scratch) explains rejection sampling →

Reinforcement Learning from Human Feedback

Rejection sampling is a sampling method that accepts or filters candidate outputs according to a specified criterion or scoring procedure.

How it is used
Deploys reward-model filtering as a simple alternative to reinforcement learning for preference fine-tuning.
How deep it goes
Explains four pipeline stages and selection strategies, but defers prompt, reward-model, sequencing, and deduplication choices to experimentation.
Examples
Uses WebGPT, Helpful and Harmless, process reward models, Llama 2 Chat, and DeepSeek R1 as pipeline examples.
What you can do after
Operate/configure a generate-rank-fine-tune pipeline and choose between per-prompt and global completion selection.

Read how Reinforcement Learning from Human Feedback explains rejection sampling →

Building LLM Applications with DSPy

Rejection sampling is a method that accepts candidate samples meeting a criterion and rejects those that do not.

How it is used
Uses rejection sampling during DSPy bootstrapping to select safe training examples as demonstrations.
How deep it goes
Describes metric-based acceptance, training-set order, and repeated rounds without formalizing the sampling method.
Examples
Uses exact-match intent labels to accept correct predictions and reject incorrect ones.
What you can do after
Apply rejection sampling with DSPy's BootstrapFewShot to compile a program using accepted training examples as demonstrations.

Read how Building LLM Applications with DSPy explains rejection sampling →

Also covered in

What to know first

Related concepts

Questions about rejection sampling

What is rejection sampling?

Rejection sampling is a method that generates candidate samples and accepts those meeting a specified criterion while discarding the others.

Which Manning books explain rejection sampling?

LLM Evaluation and Alignment, The Foundational Ideas, Fabulous Adventures in Data Structures and Algorithms, Build a DeepSeek Model (From Scratch), Reinforcement Learning from Human Feedback and Building LLM Applications with DSPy.

What should I know before learning rejection sampling?

Manning books that teach rejection sampling usually introduce chain-of-thought prompting, DeepSeek-R1-Zero, fine-tuning and policy first.

What is the difference between rejection sampling and reinforcement learning?

Rejection sampling is a method that generates candidate samples and accepts those meeting a specified criterion while discarding the others. Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.

What is rejection sampling a kind of?

Rejection sampling is a kind of data curation. Data curation is the systematic selection, review, cleaning, organization, and maintenance of data for a particular purpose.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.