liveBook · Manning

Thompson sampling

Thompson sampling is a Bayesian sequential decision method that samples from posterior beliefs about uncertain rewards or outcomes and selects an action according to the sampled values.

How 5 Manning books explain it

Experimentation for Engineers

Thompson sampling is a Bayesian bandit algorithm that samples each alternative's expected reward from its posterior distribution and selects an alternative according to those samples.

How it is used
Optimizes live A/B-style experiments by allocating more measurements to alternatives likely to improve a business metric.
How deep it goes
Works through bootstrap probability matching, online-bootstrap efficiency, stopping, and practical-significance safeguards.
Examples
Uses ad click-through-rate selection and a social-media recommender choosing posts by predicted viewing time.
What you can do after
Apply it via bootstrap sampling and an online bootstrap to select arms, stop experiments, or deploy ensemble-based contextual policies.

Read how Experimentation for Engineers explains Thompson sampling →

Bayesian Optimization in Action

Thompson sampling is a Bayesian decision policy that samples a plausible model or parameter value from a posterior distribution and selects the action that is optimal under that sample.

How it is used
Guides Bayesian optimization queries by selecting a sampled maximizer of an uncertain objective.
How deep it goes
Explains posterior-sampling intuition and a finite Sobol-candidate GP implementation, while deferring full-function sampling as computationally impossible.
Examples
Uses a biased weather coin flip, GP function draws, one-dimensional optimization progress, and batch hyperparameter tuning.
What you can do after
Apply it via BoTorch to select sequential or batch Bayesian-optimization queries from GP posterior samples.

Read how Bayesian Optimization in Action explains Thompson sampling →

Applied Reinforcement Learning

A Bayesian bandit action-selection algorithm that samples each action's expected reward from its posterior distribution and selects the action with the best sample.

How it is used
Balances exploration and exploitation when selecting ads with uncertain click-through rates.
How deep it goes
Works through Beta beliefs, sampled action selection, and click-based updates in an agent implementation.
Examples
Uses a simulated ad campaign with binary clicks to compare Thompson sampling against other bandit policies.
What you can do after
Build a Thompson sampling agent for binary ad-click rewards and evaluate its simulated regret.

Read how Applied Reinforcement Learning explains Thompson sampling →

Timeless Algorithms, The Foundational Ideas

Thompson sampling is a Bayesian sequential decision algorithm that samples from posterior distributions over alternatives and selects the alternative with the highest sampled value.

How it is used
Balances exploration and exploitation when choosing among options whose best-performing alternative is initially unknown.
How deep it goes
Explains prior-to-posterior updating and reward sampling as a repeated decision process.
Examples
Names online advertising, recommendation systems, website optimization, and pricing, with clicks, purchases, and conversions as outcomes.
What you can do after
Understand how posterior beliefs guide repeated choices while leaving uncertain alternatives opportunities for exploration.

Read how Timeless Algorithms, The Foundational Ideas explains Thompson sampling →

Grokking Bayes

Thompson sampling is a Bayesian action-selection method that samples candidate parameters or values from their posterior distribution and chooses the corresponding action.

How it is used
Balances exploitation and exploration when repeatedly choosing among uncertain actions or reward-producing arms.
How deep it goes
Explains the sampling rule and gives a small numerical example, but does not develop posterior construction or implementation details.
Examples
Uses sampled utilities for policies A and B, compares them with doing nothing, and situates the method in multi-armed bandits.
What you can do after
Apply a stochastic action-selection rule that samples plausible utilities and chooses the largest sampled value.

Read how Grokking Bayes explains Thompson sampling →

What to know first

Related concepts

Questions about Thompson sampling

What is Thompson sampling?

Thompson sampling is a Bayesian sequential decision method that samples from posterior beliefs about uncertain rewards or outcomes and selects an action according to the sampled values.

Which Manning books explain Thompson sampling?

Experimentation for Engineers, Bayesian Optimization in Action, Applied Reinforcement Learning, Timeless Algorithms, The Foundational Ideas and Grokking Bayes.

What should I know before learning Thompson sampling?

Manning books that teach Thompson sampling usually introduce multi-armed bandit, Bayesian optimization, Bayesian inference and click-through rate first.

What kinds of Thompson sampling are there?

Kinds of Thompson sampling include bootstrap Thompson sampling.

What is Thompson sampling a kind of?

Thompson sampling is a kind of multi-armed bandit. A multi-armed bandit is a sequential decision problem in which an agent repeatedly chooses among alternatives with uncertain rewards, balancing reward seeking with information gathering.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.