liveBook · Manning

Regret

Regret is the difference between the reward or utility obtained from a choice and that obtainable from the best alternative under the same circumstances, often accumulated across decisions.

How 4 Manning books explain it

Experimentation for Engineers

Regret is the lost outcome relative to the outcome that would have been obtained by choosing the best alternative throughout the decision process.

How it is used
Frames the business cost of exploration while comparing A/B testing and multi-armed bandit policies.
How deep it goes
Defines and calculates regret, compares policies numerically, and states asymptotic bounds without deriving their proofs.
Examples
Uses ad click-through rates, simulated A/B tests, epsilon-greedy runs, and decaying-epsilon comparisons.
What you can do after
Evaluate experimental policies by quantifying lost business-metric performance against the best alternative.

Read how Experimentation for Engineers explains regret →

Optimization Algorithms: AI techniques for design, planning, and control problems

Regret is the cumulative difference between the reward obtainable by always choosing the best action and the reward actually obtained.

How it is used
Measures the cost of action selection while comparing exploration and exploitation strategies in multi-armed bandits.
How deep it goes
Calculates per-trial and average regret with a worked example and Python implementations, but does not develop formal regret bounds.
Examples
Uses three slot machines, 90 trials, payoff distributions, strategy pseudocode, and average-regret outputs.
What you can do after
Diagnose bandit strategies by implementing cumulative and average regret calculations in NumPy.

Read how Optimization Algorithms: AI techniques for design, planning, and control problems explains regret →

Causal AI

Regret is the difference between the utility or reward realized by an action and the utility or reward that would have been realized by an alternative action.

How it is used
Uses counterfactual outcome differences to learn from decisions and improve future policies.
How deep it goes
Defines regret, models counterfactual uncertainty probabilistically, samples expected regret, and extends it to cumulative policy optimization.
Examples
Uses portfolio backtesting, failed deals, game simulations, player purchases, and recommendation-policy comparisons.
What you can do after
Evaluate alternative actions or policies and estimate which would minimize cumulative regret.

Read how Causal AI explains regret →

Applied Reinforcement Learning

Regret is the difference between the reward obtained by an algorithm and the reward it could have obtained by acting optimally over the same decisions.

How it is used
Quantifies missed reward to compare bandit algorithm performance in stochastic one-step decisions.
How deep it goes
Defines per-decision and cumulative regret, describes curve behavior, and notes that real-campaign regret cannot be calculated directly.
Examples
Compares ad-selection policies through cumulative regret curves over 500 iterations.
What you can do after
Evaluate bandit policies by comparing cumulative regret curves; use business KPIs as proxies when true regret cannot be calculated.

Read how Applied Reinforcement Learning explains regret →

Also covered in

What to know first

Related concepts

Questions about regret

What is regret?

Regret is the difference between the reward or utility obtained from a choice and that obtainable from the best alternative under the same circumstances, often accumulated across decisions.

Which Manning books explain regret?

Experimentation for Engineers, Optimization Algorithms: AI techniques for design, planning, and control problems, Causal AI and Applied Reinforcement Learning.

What should I know before learning regret?

Manning books that teach regret usually introduce AI agent, click-through rate, mean and multi-armed bandit first.

What kinds of regret are there?

Kinds of regret include negative regret.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.