Regret measures the reward an optimal decision would have earned minus the reward an algorithm actually earned. In ad selection, each suboptimal choice adds missed opportunity; minimizing regret means learning to choose well sooner. section 5.4
Cumulative regret reveals the cost of exploration and premature commitment. Effective policies may accumulate regret while learning, then flatten as they identify better choices; inefficient policies keep accumulating it. Since true regret cannot be calculated in real campaigns, business KPIs serve as proxies. section 5.4 section 5.7
Figure 5.8 Cumulative Regret comparisons of different policies for 500 iterations (for better comparison).
Compare cumulative regret across policies over 500 iterations. section 5.4



