liveBook · Manning

Combinatorial bandit

As explained in Applied Reinforcement Learning

A combinatorial bandit is a bandit model in which each decision selects a combination or set of actions whose joint outcome determines the payoff.

A combinatorial bandit handles decisions where several actions are selected together and the payoff depends on the complete combination rather than one isolated action. section 5.6 section 5.7

A slate of recommendations

A homepage may fill several slots, a carousel may show ten product tiles, or a feed may rank items from top to bottom. Each complete slate determines the outcome, so the decision is about optimizing the set while avoiding five near-identical recommendations. section 5.6

Use the model when:

  • Payoff depends on slates, bundles, or rankings rather than single actions. section 5.7
  • Optimization must balance performance with diversity so selected items are not near-duplicates. section 5.6
  • The action space is large enough that slate optimization must be performed efficiently. section 5.6

How Applied Reinforcement Learning covers it

How deep it goes
Defines joint payoffs and motivates efficient optimization and diversity, but does not detail specific algorithms.
Examples
Uses homepage slots, carousels, product tiles, feeds, slates, bundles, and rankings as concrete decision examples.
What you can do after
Evaluate whether a decision problem requires optimizing a set of actions instead of selecting one arm.
How it connects
Connects to contextual bandits, reward shaping, constrained bandits, exploration, and diversity optimization.

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.