Thompson sampling
Thompson sampling is a Bayesian sequential decision method that samples from posterior beliefs about uncertain rewards or outcomes and selects an action according to the sampled values.
Experimentation for Engineers
Thompson sampling is a Bayesian bandit algorithm that samples each alternative's expected reward from its posterior distribution and selects an alternative according to those samples.
- How it is used
- Optimizes live A/B-style experiments by allocating more measurements to alternatives likely to improve a business metric.
- How deep it goes
- Works through bootstrap probability matching, online-bootstrap efficiency, stopping, and practical-significance safeguards.
- Examples
- Uses ad click-through-rate selection and a social-media recommender choosing posts by predicted viewing time.
- What you can do after
- Apply it via bootstrap sampling and an online bootstrap to select arms, stop experiments, or deploy ensemble-based contextual policies.
Read how Experimentation for Engineers explains Thompson sampling →
Bayesian Optimization in Action
Thompson sampling is a Bayesian decision policy that samples a plausible model or parameter value from a posterior distribution and selects the action that is optimal under that sample.
- How it is used
- Guides Bayesian optimization queries by selecting a sampled maximizer of an uncertain objective.
- How deep it goes
- Explains posterior-sampling intuition and a finite Sobol-candidate GP implementation, while deferring full-function sampling as computationally impossible.
- Examples
- Uses a biased weather coin flip, GP function draws, one-dimensional optimization progress, and batch hyperparameter tuning.
- What you can do after
- Apply it via BoTorch to select sequential or batch Bayesian-optimization queries from GP posterior samples.
Read how Bayesian Optimization in Action explains Thompson sampling →
Applied Reinforcement Learning
A Bayesian bandit action-selection algorithm that samples each action's expected reward from its posterior distribution and selects the action with the best sample.
- How it is used
- Balances exploration and exploitation when selecting ads with uncertain click-through rates.
- How deep it goes
- Works through Beta beliefs, sampled action selection, and click-based updates in an agent implementation.
- Examples
- Uses a simulated ad campaign with binary clicks to compare Thompson sampling against other bandit policies.
- What you can do after
- Build a Thompson sampling agent for binary ad-click rewards and evaluate its simulated regret.
Read how Applied Reinforcement Learning explains Thompson sampling →
Timeless Algorithms, The Foundational Ideas
Thompson sampling is a Bayesian sequential decision algorithm that samples from posterior distributions over alternatives and selects the alternative with the highest sampled value.
- How it is used
- Balances exploration and exploitation when choosing among options whose best-performing alternative is initially unknown.
- How deep it goes
- Explains prior-to-posterior updating and reward sampling as a repeated decision process.
- Examples
- Names online advertising, recommendation systems, website optimization, and pricing, with clicks, purchases, and conversions as outcomes.
- What you can do after
- Understand how posterior beliefs guide repeated choices while leaving uncertain alternatives opportunities for exploration.
Read how Timeless Algorithms, The Foundational Ideas explains Thompson sampling →
Grokking Bayes
Thompson sampling is a Bayesian action-selection method that samples candidate parameters or values from their posterior distribution and chooses the corresponding action.
- How it is used
- Balances exploitation and exploration when repeatedly choosing among uncertain actions or reward-producing arms.
- How deep it goes
- Explains the sampling rule and gives a small numerical example, but does not develop posterior construction or implementation details.
- Examples
- Uses sampled utilities for policies A and B, compares them with doing nothing, and situates the method in multi-armed bandits.
- What you can do after
- Apply a stochastic action-selection rule that samples plausible utilities and chooses the largest sampled value.
What to know first
- multi-armed banditA multi-armed bandit is a sequential decision problem in which an agent repeatedly chooses among alternatives with uncertain rewards, balancing reward seeking with information gathering.
- Bayesian optimizationBayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.
- Bayesian inferenceBayesian inference updates probability distributions over hypotheses or unknown quantities by combining prior information with observed evidence and a likelihood model.
- click-through rateClick-through rate is the proportion of impressions or opportunities to display an item that result in a click on that item.
Related concepts
- Bayes' theoremuses · Bayes' theorem relates conditional probabilities, expressing a posterior probability in terms of a prior probability, a likelihood, and the evidence.
- MaxPosteriorSamplingimplemented by · MaxPosteriorSampling is a BoTorch sampler that draws posterior samples at candidate points and selects candidates maximizing those samples.
- multi-armed banditkind of · A multi-armed bandit is a sequential decision problem in which an agent repeatedly chooses among alternatives with uncertain rewards, balancing reward seeking with information gathering.
- exploration-exploitation trade-offaddresses · The exploration-exploitation trade-off is the decision problem of balancing actions that gather information about uncertain alternatives with actions chosen for their currently expected payoff.
- beta distributionuses · The beta distribution is a continuous probability distribution on the interval from zero to one, parameterized by two positive shape parameters.
- bootstrap Thompson samplingincludes · Bootstrap Thompson sampling is a form of Thompson sampling that estimates each alternative's probability of being best from bootstrap resamples or an ensemble of fitted models.
Questions about Thompson sampling
What is Thompson sampling?
Thompson sampling is a Bayesian sequential decision method that samples from posterior beliefs about uncertain rewards or outcomes and selects an action according to the sampled values.
Which Manning books explain Thompson sampling?
Experimentation for Engineers, Bayesian Optimization in Action, Applied Reinforcement Learning, Timeless Algorithms, The Foundational Ideas and Grokking Bayes.
What should I know before learning Thompson sampling?
Manning books that teach Thompson sampling usually introduce multi-armed bandit, Bayesian optimization, Bayesian inference and click-through rate first.
What kinds of Thompson sampling are there?
Kinds of Thompson sampling include bootstrap Thompson sampling.
What is Thompson sampling a kind of?
Thompson sampling is a kind of multi-armed bandit. A multi-armed bandit is a sequential decision problem in which an agent repeatedly chooses among alternatives with uncertain rewards, balancing reward seeking with information gathering.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




