Return

As explained in Grokking AI Algorithms, Second Edition

Cumulative reward is the sum of reward signals accumulated across multiple actions or attempts.

Return is the total reward accumulated across a sequence of actions or attempts. It captures the overall result, not just whether one individual action was good or bad. section 10.3 section 10.8

This perspective is useful when actions build toward a larger goal. A move that seems unfavorable now may still improve the eventual outcome, so the agent must balance immediate rewards with long-term rewards. section 10.3 section 10.7

Measuring a walking agent

Suppose an agent receives penalties when it falls or takes strides that are too long. Add those reward signals across its attempts to obtain cumulative reward. To measure average reward per action, divide that cumulative reward by the total number of actions. This reveals whether the agent generally chooses good actions, even when it does not complete the final goal. section 10.8

Figure 10.5 The self-driving car in the Parking-Lot Problem

Use the parking-lot scene to see why the combined consequences of movement, collisions, and reaching the owner matter more than any single move. section 10.3

How Grokking AI Algorithms, Second Edition covers it

How deep it goes

Defines cumulative reward and calculates average reward per action, while deferring detailed return formulas and theory.

Examples

Uses chess planning, a self-driving parking-lot problem, and a walking agent with penalties.

What you can do after

Evaluate reinforcement-learning performance by comparing cumulative reward and average reward per action.

How it connects

Connects return with reinforcement learning, reward, actions, environments, Q-tables, and long-term rewards.

liveBookmanning.com

Read Grokking AI Algorithms, Second Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsThe total reward accumulated by an agent over a sequence of decisions, often considered in expectation.
• 

  liveBookmanning.com
  Bayesian Optimization in ActionCumulative reward is the total reward accumulated across a sequence of actions or time steps.
• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackCumulative reward is the aggregate of rewards collected across an episode or other interaction sequence.

Compare how 4 books explain return →

Related concepts

• reward signalcontrasts with · A reward signal is an evaluative feedback value representing the desirability or payoff of an action, outcome, or state.
• value functionmeasured by · A value function assigns or estimates the expected cumulative reward, utility, or quality associated with a state or action under a policy or decision rule.

This explainer was generated by Manning's knowledge graph from the text of Grokking AI Algorithms, Second Edition. The section links point to the parts of the book it draws on.