Offline evaluation tests a bandit on historical interactions when a live experiment is unavailable. Replay each customer's context and let the agent choose a discount. Count the historical reward only when its choice matches the recorded discount; otherwise, record zero because the alternative reward is unknown. section 5.5
A practical evaluation separates learning from testing:
- Train LinUCB on the first 80% of the data by updating it with historical context-action-reward tuples. section 5.5
- Evaluate on the remaining 20%, comparing the pretrained contextual agent with a cold-start epsilon-greedy baseline. section 5.5
Personalized discounts
For each customer, an agent selects a discount. If it selects the discount given historically, the recorded reward contributes to its measured performance; a different choice receives zero. This conservative rule cannot reveal what the customer would have done under another discount. section 5.5
Figure 5.11 Performance comparison on the test set.
Compare the agents' cumulative rewards on the held-out test data. section 5.5




