Bootstrap Thompson sampling selects among alternatives by fitting or resampling multiple plausible models, then treating each model’s predicted best alternative as a probability sample. Alternatives chosen by more models receive more traffic, while disagreement preserves targeted exploration. section 3.3.2 section 5.3.2
Ten recommender models
Suppose ten equally credible models predict which post will maximize viewing time. If seven select post #1 and three select post #2, the policy displays post #1 with probability 7/10 and post #2 with probability 3/10. Drawing a fresh model for each user therefore makes action probability equal to the estimated probability of being best. section 5.3.2
A practical implementation follows this pattern:
- Fit an ensemble of models from bootstrap-resampled observations, using a suitable supervised-learning method. section 5.3.1 section 5.4
- For each decision, draw one model uniformly at random and choose the action with its highest predicted reward. section 5.3.2
- For non-contextual metrics such as click-through rate, bootstrap each arm’s observations, compare sampled estimates, and select the largest. section 3.3.2
- Replace repeated full-data resampling with an online bootstrap when decision latency matters; otherwise repeated resampling can grow as O(T²). section 3.3.2
