Regret
Regret is the difference between the reward or utility obtained from a choice and that obtainable from the best alternative under the same circumstances, often accumulated across decisions.
Experimentation for Engineers
Regret is the lost outcome relative to the outcome that would have been obtained by choosing the best alternative throughout the decision process.
- How it is used
- Frames the business cost of exploration while comparing A/B testing and multi-armed bandit policies.
- How deep it goes
- Defines and calculates regret, compares policies numerically, and states asymptotic bounds without deriving their proofs.
- Examples
- Uses ad click-through rates, simulated A/B tests, epsilon-greedy runs, and decaying-epsilon comparisons.
- What you can do after
- Evaluate experimental policies by quantifying lost business-metric performance against the best alternative.
Optimization Algorithms: AI techniques for design, planning, and control problems
Regret is the cumulative difference between the reward obtainable by always choosing the best action and the reward actually obtained.
- How it is used
- Measures the cost of action selection while comparing exploration and exploitation strategies in multi-armed bandits.
- How deep it goes
- Calculates per-trial and average regret with a worked example and Python implementations, but does not develop formal regret bounds.
- Examples
- Uses three slot machines, 90 trials, payoff distributions, strategy pseudocode, and average-regret outputs.
- What you can do after
- Diagnose bandit strategies by implementing cumulative and average regret calculations in NumPy.
Causal AI
Regret is the difference between the utility or reward realized by an action and the utility or reward that would have been realized by an alternative action.
- How it is used
- Uses counterfactual outcome differences to learn from decisions and improve future policies.
- How deep it goes
- Defines regret, models counterfactual uncertainty probabilistically, samples expected regret, and extends it to cumulative policy optimization.
- Examples
- Uses portfolio backtesting, failed deals, game simulations, player purchases, and recommendation-policy comparisons.
- What you can do after
- Evaluate alternative actions or policies and estimate which would minimize cumulative regret.
Applied Reinforcement Learning
Regret is the difference between the reward obtained by an algorithm and the reward it could have obtained by acting optimally over the same decisions.
- How it is used
- Quantifies missed reward to compare bandit algorithm performance in stochastic one-step decisions.
- How deep it goes
- Defines per-decision and cumulative regret, describes curve behavior, and notes that real-campaign regret cannot be calculated directly.
- Examples
- Compares ad-selection policies through cumulative regret curves over 500 iterations.
- What you can do after
- Evaluate bandit policies by comparing cumulative regret curves; use business KPIs as proxies when true regret cannot be calculated.
Also covered in
What to know first
- AI agentAn AI agent is a computational system that uses artificial intelligence to interpret information and select or perform actions toward goals while interacting with an environment.
- click-through rateClick-through rate is the proportion of impressions or opportunities to display an item that result in a click on that item.
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
- multi-armed banditA multi-armed bandit is a sequential decision problem in which an agent repeatedly chooses among alternatives with uncertain rewards, balancing reward seeking with information gathering.
Related concepts
- study designproduced by · Study design is the planned arrangement of procedures, conditions, measurements, and comparisons for investigating a question and supporting valid conclusions.
- reward signalmeasures · A reward signal is a value or feedback indicating the desirability, benefit, or cost associated with an action, outcome, or behavior.
- exploration-exploitation trade-offmeasures · The exploration-exploitation trade-off is the decision problem of balancing actions that gather information about uncertain alternatives with actions chosen for their currently expected payoff.
- minimax regret methodused by · A decision method that selects the alternative with the smallest maximum regret relative to the best alternative under each possible scenario.
Questions about regret
What is regret?
Regret is the difference between the reward or utility obtained from a choice and that obtainable from the best alternative under the same circumstances, often accumulated across decisions.
Which Manning books explain regret?
Experimentation for Engineers, Optimization Algorithms: AI techniques for design, planning, and control problems, Causal AI and Applied Reinforcement Learning.
What should I know before learning regret?
Manning books that teach regret usually introduce AI agent, click-through rate, mean and multi-armed bandit first.
What kinds of regret are there?
Kinds of regret include negative regret.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



