Exploration deliberately tries actions with uncertain payoffs to gather information. It may sacrifice short-term reward, but can improve estimates and reveal better choices. Because actions shape the data available for learning, exploration must be balanced with exploitation: choosing the option that current knowledge favors. section 5.2
Choosing what to offer
A business can test less-known ad creatives, discounts, or product bundles to learn how they perform. Always choosing the current favorite may miss a better option; testing too many weak options can waste immediate rewards. section 5.2
Common strategies direct exploration in different ways:
- ε-greedy sometimes selects a random action to gather data. section 5.2
- UCB favors actions that are promising and uncertain. section 5.2
- Thompson Sampling uses a Bayesian approach to balance exploration and exploitation. section 5.2
- Monte Carlo Tree Search balances trying new branches with focusing on promising paths. section 7.5
