liveBook · Manning

Thompson sampling

As explained in Timeless Algorithms, The Foundational Ideas

Thompson sampling is a Bayesian sequential decision algorithm that samples from posterior distributions over alternatives and selects the alternative with the highest sampled value.

When the best option is unknown, Thompson sampling uses Bayesian beliefs to choose among alternatives while learning from their observed rewards. section 2.4.1

At each decision point:

  • Maintain a prior distribution for each option’s expected reward, then update it to a posterior as outcomes arrive. section 2.4.1
  • Sample a reward from each option’s posterior and choose the option with the highest sampled value. section 2.4.1
  • Repeat as evidence accumulates; increasingly precise posteriors concentrate decisions on the best-performing option. section 2.4.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains prior-to-posterior updating and reward sampling as a repeated decision process.
Examples
Names online advertising, recommendation systems, website optimization, and pricing, with clicks, purchases, and conversions as outcomes.
What you can do after
Understand how posterior beliefs guide repeated choices while leaving uncertain alternatives opportunities for exploration.
How it connects
Connects Bayesian inference, Bayes’ theorem, posterior distributions, and the exploration–exploitation tradeoff.

Also explained in

Compare how 5 books explain Thompson sampling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.