liveBook · Manning

Restaurant scheduling problem

As explained in Applied Reinforcement Learning

A finite-horizon decision problem in which a restaurant chooses when to seat parties to maximize expected revenue under uncertain arrivals.

With one premium table and four hours until closing, choose between seating an available small party for $50 or waiting for a large party worth $120, which arrives with 30% probability each hour. The goal is to maximize expected evening revenue. section 6.2

The decision model maps the restaurant operation to reinforcement-learning elements:

  • State: hours remaining, from one to four. section 6.2
  • Actions: seat the small party now or wait for a large party. section 6.2
  • Rewards and transitions: seating a small party earns $50; waiting may earn $120 or $0, and can leave another hour to decide. section 6.2
  • A discount factor of 0.9 is used when valuing future revenue. section 6.2

Why the best choice depends on time

With one hour left, waiting is worth 0.3 × $120 = $36, so seating immediately is better. With two hours left, waiting is worth $36 + 0.7 × 0.9 × $50 = $67.50, so waiting is better. section 6.2

The analytically calculated state values provide a benchmark for temporal-difference learning, which estimates values through experience without knowing the arrival probability or having the analytical solution. Example learned values differ from the benchmark, so estimates need not match exactly. section 6.2

How Applied Reinforcement Learning covers it

How deep it goes
Works through one- and two-hour value calculations and backward induction, then compares learned estimates with analytical values.
Examples
Uses a premium table, $50 small parties, $120 large parties arriving with 30% probability, and a four-hour horizon.
What it assumes
Assumes familiarity with dynamic-programming value iteration or policy iteration while defining the restaurant decision setup directly.
What you can do after
Evaluate learned state values against analytical optimal values in a small uncertain scheduling problem.
How it connects
Connects temporal-difference learning with dynamic programming, policy iteration, and value iteration.

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.