liveBook · Manning

Importance sampling

Importance sampling is a statistical technique that estimates expectations under a target distribution by reweighting samples drawn from another distribution.

How 3 Manning books explain it

Reinforcement Learning from Human Feedback

Importance sampling is a method for estimating an expectation under one probability distribution using samples drawn from another distribution and probability weights.

How it is used
Corrects distribution mismatches when reusing policy rollouts and training with separate sampler and learner systems.
How deep it goes
Derives the expectation identity and policy ratios, then develops clipping, truncation, objectives, and an implementation.
Examples
Uses reused rollout batches, sampler-versus-learner token probabilities, and a PyTorch per-token loss calculation.
What you can do after
Apply it via PyTorch to cap sampler-learner weights and adjust a per-token policy-gradient loss.

Read how Reinforcement Learning from Human Feedback explains importance sampling →

Machine Learning Algorithms in Depth

Importance sampling is a Monte Carlo technique that estimates expectations or integrals under a target distribution by weighting samples drawn from a proposal distribution.

How it is used
Estimates target-distribution expectations and integrals by sampling an easier proposal distribution.
How deep it goes
Derives weighted Monte Carlo estimation and implements sampling, filtering, density evaluation, and averaging, while deferring further normalization details.
Examples
Uses a sinusoidal integrand, an unnormalized chi target, a scaled Gaussian proposal, positive filtering, and numerical ground truth.
What you can do after
Build it from scratch by coding a proposal sampler, importance weights, filtered samples, and a weighted Monte Carlo estimate.

Read how Machine Learning Algorithms in Depth explains importance sampling →

Causal AI

Importance sampling estimates expectations under a target distribution using samples from another distribution and likelihood-based weights.

How it is used
Supports probabilistic inference when conditioning would discard many generated samples.
How deep it goes
Demonstrates weighted Monte Carlo inference and a Pyro workflow, but defers broader inference algorithms and causal applications.
Examples
Uses a Pyro transportation model to estimate education probabilities for train riders.
What you can do after
Apply it via Pyro to condition a model, sample an empirical marginal, and estimate target probabilities.

Read how Causal AI explains importance sampling →

What to know first

Related concepts

Questions about importance sampling

What is importance sampling?

Importance sampling is a statistical technique that estimates expectations under a target distribution by reweighting samples drawn from another distribution.

Which Manning books explain importance sampling?

Reinforcement Learning from Human Feedback, Machine Learning Algorithms in Depth and Causal AI.

What should I know before learning importance sampling?

Manning books that teach importance sampling usually introduce conditional probability and parameter first.

What is importance sampling a kind of?

Importance sampling is a kind of probabilistic reasoning. Probabilistic reasoning is drawing conclusions or making decisions under uncertainty by representing and evaluating possibilities using probabilities.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.