liveBook · Manning

Optimistic prior

As explained in Applied Reinforcement Learning

An optimistic prior assigns favorable initial beliefs to uncertain actions or parameters before sufficient observations are available.

When a bandit launches a new option with little or no data, an optimistic prior gives that option the benefit of the doubt. Rather than letting established choices dominate immediately, the system treats the newcomer as promising enough to receive a fair chance. section 5.6

Use optimistic priors to address cold-start decisions such as:

  • Launching a new advertisement or creative before its performance is known. section 5.6
  • Serving a newly encountered customer type when similar historical observations are unavailable. section 5.6
  • Preventing incumbent options from burying alternatives before they have accumulated evidence. section 5.6

A fair launch for a new creative

Suppose a contextual bandit already knows how several advertisements perform but has no observations for a newly launched creative. An optimistic prior initially assumes the newcomer may perform well enough to merit exposure. Its subsequent rewards can then replace that favorable starting belief with evidence. A warm start offers an alternative by borrowing knowledge from similar ads or users. section 5.6 section 5.7

How Applied Reinforcement Learning covers it

How deep it goes
Defines the intuition and use case but does not specify a prior distribution, update rule, or implementation algorithm.
Examples
Uses new advertisements, new customer types, and similar ads or users as cold-start examples.
What it assumes
Assumes familiarity with bandit actions or arms and introduces the cold-start motivation through business examples.
What you can do after
Apply it to give newly launched ads or customer segments a fair chance during bandit cold starts.
How it connects
Connects cold start, warm starts, contextual bandits, exploration, and covariate shift.

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.