Return

As explained in Bayesian Optimization in Action

Cumulative reward is the total reward accumulated across a sequence of actions or time steps.

Return is the total reward accumulated across a sequence of actions or time steps. In a multi-armed bandit, it is the sum of rewards received during a fixed number of pulls: maximize frequent successful rewards, not merely one especially good outcome. section 5.5 section 5.1.1

Return shapes decision-making under uncertainty:

• Balance exploration of uncertain actions with exploitation of actions that already perform well. section 5.1.1
• Use past rewards to choose the next action while accounting for a limited number of opportunities. section 5.5
• Distinguish cumulative reward from simple reward: bandits maximize the sum, whereas BayesOpt maximizes the largest observed objective value. section 5.1.2

Casino slot machines

Suppose 100 pulls produce 70 coins from one machine. Its observed success rate is 70%, but an uncertain machine with no observed successes may still deserve exploration if it could have a higher reward rate. The policy seeks the greatest total coin count across all pulls. section 5.1.1

Figure 5.3 An example of an MAB dataset showing the exploration–exploitation dilemma. An MAB policy must choose between a machine with a consistently high success rate and one with a reward rate that is uncertain.

Use the dataset illustration to compare a consistently successful machine with one whose reward rate remains uncertain. section 5.1.1

How Bayesian Optimization in Action covers it

How deep it goes

Defines cumulative reward, contrasts it with simple reward, and illustrates its exploration–exploitation implications through slot-machine data.

Examples

Uses casino slot machines, coin counts, finite pulls, and a 70-successes-in-100-pulls dataset.

What you can do after

Evaluate bandit policies by how effectively they accumulate rewards while balancing exploration and exploitation.

How it connects

Connects multi-armed bandits with Bayes optimization, exploration–exploitation, UCB, and Thompson sampling.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsThe total reward accumulated by an agent over a sequence of decisions, often considered in expectation.
• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackCumulative reward is the aggregate of rewards collected across an episode or other interaction sequence.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionCumulative reward is the sum of reward signals accumulated across multiple actions or attempts.

Compare how 4 books explain return →

Related concepts

• reward signalcontrasts with · A reward signal is an evaluative feedback value representing the desirability or payoff of an action, outcome, or state.
• value functionmeasured by · A value function assigns or estimates the expected cumulative reward, utility, or quality associated with a state or action under a policy or decision rule.
• Monte Carlo simulationproduced by · Monte Carlo simulation is a computational technique that uses repeated random sampling to estimate quantities or characterize possible outcomes.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.