liveBook · Manning

Cold start problem

As explained in Applied Reinforcement Learning

The cold start problem is the difficulty of making effective recommendations or decisions when a new action, item, user, or system has little or no interaction data.

A new ad creative or customer type has little or no interaction data, leaving a bandit unsure how to act. Without a fair chance to learn, new options can be buried by established ones. section 5.6

Give new options a chance with:

  • Optimistic priors that give new arms the benefit of the doubt. section 5.6
  • Warm starts that borrow knowledge from similar arms or users. section 5.6

How Applied Reinforcement Learning covers it

How deep it goes
Recommends optimistic priors and warm starts but does not detail their implementation.
Examples
Uses a new ad creative and a new customer type as cold-start cases.
What you can do after
Apply optimistic priors or warm starts to give new options a fair chance.
How it connects
Connects cold starts to bandits, exploration, and personalization.

Also explained in

Compare how 7 books explain cold start problem →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.