liveBook · Manning

Cumulative reward

As explained in Timeless Algorithms, The Foundational Ideas

Expected cumulative reward is the probability-weighted expected total of rewards accumulated over a decision process.

Return is the expected total reward accumulated across a decision process. In a Markov decision process, choosing an action means considering its immediate reward alongside the probability-weighted value of possible future states—not just the immediate reward. section 6.5.1

This expectation matters when an action can lead to several possible next states. The Bellman equation provides the recursive relationship for combining immediate reward with expected future value. section 6.5.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains its role in an MDP objective and links it to expected future value, without giving a return formula.
Examples
Uses inventory levels, network locations, and available resources as examples of decision states.
What you can do after
Evaluate actions by weighing immediate rewards against probability-weighted future values.
How it connects
Connects return to Markov decision processes, rewards, transition probabilities, and the Bellman equation.

Also explained in

Compare how 5 books explain cumulative reward →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.