With one premium table and four hours until closing, choose between seating an available small party for $50 or waiting for a large party worth $120, which arrives with 30% probability each hour. The goal is to maximize expected evening revenue. section 6.2
The decision model maps the restaurant operation to reinforcement-learning elements:
- State: hours remaining, from one to four. section 6.2
- Actions: seat the small party now or wait for a large party. section 6.2
- Rewards and transitions: seating a small party earns $50; waiting may earn $120 or $0, and can leave another hour to decide. section 6.2
- A discount factor of 0.9 is used when valuing future revenue. section 6.2
Why the best choice depends on time
With one hour left, waiting is worth 0.3 × $120 = $36, so seating immediately is better. With two hours left, waiting is worth $36 + 0.7 × 0.9 × $50 = $67.50, so waiting is better. section 6.2
The analytically calculated state values provide a benchmark for temporal-difference learning, which estimates values through experience without knowing the arrival probability or having the analytical solution. Example learned values differ from the benchmark, so estimates need not match exactly. section 6.2
