Regret

Q: How deeply does Optimization Algorithms: AI techniques for design, planning, and control problems cover regret?

Calculates per-trial and average regret with a worked example and Python implementations, but does not develop formal regret bounds.

As explained in Optimization Algorithms: AI techniques for design, planning, and control problems

Regret is the cumulative difference between the reward obtainable by always choosing the best action and the reward actually obtained.

Regret measures the cumulative reward lost by choosing actions instead of always selecting the best available action. In multi-armed bandits, it exposes the cost of exploration and poor exploitation. section 12.1.6

Three slot machines over 90 trials

Suppose three machines have mean payoffs of 8, 10, and 5. Choosing each machine 30 times produces 690 expected reward, while always choosing machine 2 produces 900. Regret is therefore 900 − 690 = 210. section 12.1.6

Compare strategies by accumulating per-trial regret:

Explore-only samples machines randomly, while exploit-only greedy commits to the currently highest-reward choice. section 12.1.6

How Optimization Algorithms: AI techniques for design, planning, and control problems covers it

Cover of Optimization Algorithms: AI techniques for design, planning, and control problems

liveBook manning.com

Read Optimization Algorithms: AI techniques for design, planning, and control problems on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Experimentation for EngineersRegret is the lost outcome relative to the outcome that would have been obtained by choosing the best alternative throughout the decision process.
liveBook manning.com

Causal AIRegret is the difference between the utility or reward realized by an action and the utility or reward that would have been realized by an alternative action.

Compare how 3 books explain regret →

Related concepts

study designproduced by · Study design is the planned arrangement of procedures, conditions, measurements, and comparisons used to investigate a question and support valid conclusions.
reward signalmeasures · A reward signal is an evaluative feedback value representing the desirability or payoff of an action, outcome, or state.
minimax regret methodused by · A decision method that selects the alternative with the smallest maximum regret relative to the best alternative under each possible scenario.
exploration-exploitation trade-offmeasures · The exploration-exploitation trade-off is the decision problem of balancing actions that gather information about uncertain alternatives with actions chosen for their currently expected payoff.

This explainer was generated by Manning's knowledge graph from the text of Optimization Algorithms: AI techniques for design, planning, and control problems. The section links point to the parts of the book it draws on.