Return
A return is the total reward accumulated over a sequence of decisions or time steps, often discounted and sometimes expressed as an expectation.
Optimization Algorithms: AI techniques for design, planning, and control problems
The total reward accumulated by an agent over a sequence of decisions, often considered in expectation.
- How it is used
- Measures the reward an agent seeks to accumulate across decisions when learning a policy or choosing bandit arms.
- How deep it goes
- Defines cumulative reward through sequential decisions and expected policy performance; leaves an explicit return equation and discounting unstated.
- Examples
- Uses three slot machines with specified payoff distributions and a 90-trial NumPy simulation comparing strategies by average regret.
- What you can do after
- Evaluate/understand cumulative reward as an RL objective and distinguish a bandit’s accumulated payoff from its measured average regret.
Bayesian Optimization in Action
Cumulative reward is the total reward accumulated across a sequence of actions or time steps.
- How it is used
- Optimizes total reward across repeated actions under a finite interaction budget.
- How deep it goes
- Defines cumulative reward, contrasts it with simple reward, and illustrates its exploration–exploitation implications through slot-machine data.
- Examples
- Uses casino slot machines, coin counts, finite pulls, and a 70-successes-in-100-pulls dataset.
- What you can do after
- Evaluate bandit policies by how effectively they accumulate rewards while balancing exploration and exploitation.
Reinforcement Learning from Human Feedback
Cumulative reward is the aggregate of rewards collected across an episode or other interaction sequence.
- How it is used
- Evaluates sequential decisions by aggregating immediate and future rewards for policy optimization and value estimation.
- How deep it goes
- Derives discounted and recursive forms, rollout estimators, and token-level PPO targets, stopping short of generalized advantage estimation details.
- Examples
- Uses CartPole episodes, sampled trajectories, and reverse-time token-level reward accumulation as concrete demonstrations.
- What you can do after
- Apply it via rollout sums and recursive updates to estimate objectives, value targets, and advantage signals.
Read how Reinforcement Learning from Human Feedback explains return →
Grokking AI Algorithms, Second Edition
Cumulative reward is the sum of reward signals accumulated across multiple actions or attempts.
- How it is used
- Frames reinforcement learning around the cumulative consequences of an agent’s actions.
- How deep it goes
- Defines cumulative reward and calculates average reward per action, while deferring detailed return formulas and theory.
- Examples
- Uses chess planning, a self-driving parking-lot problem, and a walking agent with penalties.
- What you can do after
- Evaluate reinforcement-learning performance by comparing cumulative reward and average reward per action.
Read how Grokking AI Algorithms, Second Edition explains return →
Related concepts
- reward signalcontrasts with · A reward signal is an evaluative feedback value representing the desirability or payoff of an action, outcome, or state.
- value functionmeasured by · A value function assigns or estimates the expected cumulative reward, utility, or quality associated with a state or action under a policy or decision rule.
- Monte Carlo simulationproduced by · Monte Carlo simulation is a computational technique that uses repeated random sampling to estimate quantities or characterize possible outcomes.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.