liveBook · Manning

Exploration

As explained in Applied Reinforcement Learning

Exploration is the selection of actions whose outcomes are uncertain in order to gather information about their payoffs.

Exploration deliberately tries actions with uncertain payoffs to gather information. It may sacrifice short-term reward, but can improve estimates and reveal better choices. Because actions shape the data available for learning, exploration must be balanced with exploitation: choosing the option that current knowledge favors. section 5.2

Choosing what to offer

A business can test less-known ad creatives, discounts, or product bundles to learn how they perform. Always choosing the current favorite may miss a better option; testing too many weak options can waste immediate rewards. section 5.2

Common strategies direct exploration in different ways:

  • ε-greedy sometimes selects a random action to gather data. section 5.2
  • UCB favors actions that are promising and uncertain. section 5.2
  • Thompson Sampling uses a Bayesian approach to balance exploration and exploitation. section 5.2
  • Monte Carlo Tree Search balances trying new branches with focusing on promising paths. section 7.5

How Applied Reinforcement Learning covers it

How deep it goes
Explains the trade-off and distinguishes random, uncertainty-directed, and Bayesian exploration strategies.
Examples
Uses ad creatives, discounts, product bundles, and search-tree branches to illustrate uncertain choices.
What you can do after
Evaluate how action choices can improve payoff estimates while earning rewards.
How it connects
Connects exploration to exploitation, multi-armed and contextual bandits, reinforcement learning, and Monte Carlo tree search.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.