Thompson sampling
Thompson sampling is a Bayesian sequential decision method that samples from posterior beliefs about uncertain rewards or outcomes and selects an action according to the sampled values.
Experimentation for Engineers
Thompson sampling is a Bayesian bandit algorithm that samples each alternative's expected reward from its posterior distribution and selects an alternative according to those samples.
- How it is used
- Optimizes live A/B-style experiments by allocating more measurements to alternatives likely to improve a business metric.
- How deep it goes
- Works through bootstrap probability matching, online-bootstrap efficiency, stopping, and practical-significance safeguards.
- Examples
- Uses ad click-through-rate selection and a social-media recommender choosing posts by predicted viewing time.
- What you can do after
- Apply it via bootstrap sampling and an online bootstrap to select arms, stop experiments, or deploy ensemble-based contextual policies.
Read how Experimentation for Engineers explains Thompson sampling →
Bayesian Optimization in Action
Thompson sampling is a Bayesian decision policy that samples a plausible model or parameter value from a posterior distribution and selects the action that is optimal under that sample.
- How it is used
- Guides Bayesian optimization queries by selecting a sampled maximizer of an uncertain objective.
- How deep it goes
- Explains posterior-sampling intuition and a finite Sobol-candidate GP implementation, while deferring full-function sampling as computationally impossible.
- Examples
- Uses a biased weather coin flip, GP function draws, one-dimensional optimization progress, and batch hyperparameter tuning.
- What you can do after
- Apply it via BoTorch to select sequential or batch Bayesian-optimization queries from GP posterior samples.
Read how Bayesian Optimization in Action explains Thompson sampling →
Related concepts
- MaxPosteriorSamplingimplemented by · MaxPosteriorSampling is a BoTorch sampler that draws posterior samples at candidate points and selects candidates maximizing those samples.
- multi-armed banditkind of · A multi-armed bandit is a sequential decision problem in which an agent chooses among options with uncertain rewards, balancing reward seeking with gathering information about those options.
- Bayes' theoremuses · Bayes' theorem is a probability rule that expresses a conditional probability in terms of a prior probability, a likelihood, and the evidence.
- exploration-exploitation trade-offaddresses · The exploration-exploitation trade-off is the decision problem of balancing actions that gather information about uncertain alternatives with actions chosen for their currently expected payoff.
- epsilon-greedycontrasts with · Epsilon-greedy is an action-selection strategy that chooses a random action with probability epsilon and otherwise selects the currently highest-valued action.
- randomized probability matchinguses · Randomized probability matching is a decision rule that selects each alternative with probability equal to the estimated probability that the alternative is optimal.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.