liveBook · Manning

Reward signal

As explained in Timeless Algorithms, The Foundational Ideas

A reward is a numerical consequence associated with an action or state transition that represents its desirability in a decision process.

In a Markov decision process (MDP), a reward is the immediate value or cost associated with an action and its outcome. It describes desirability, while transition probabilities describe which next states may follow. section 6.5.1

An appealing immediate reward may lead to less desirable future states, so decisions aim to maximize expected cumulative reward rather than reward at one step. When rewards or transitions are unknown, reinforcement learning uses observed rewards and outcomes to improve decisions. section 6.5.1 section 6.5.3

Inventory and navigation

An inventory manager weighs an order’s immediate consequences against possible future inventory states. A robot in an unfamiliar environment observes a reward after acting and uses that experience to improve later decisions. section 6.5.1 section 6.5.3

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines immediate reward and explains its role in cumulative evaluation, but does not specify a reward-design procedure.
Examples
Uses inventory ordering under uncertain demand and a robot learning to navigate an unfamiliar environment.
What you can do after
Evaluate decisions by considering immediate rewards alongside the value of possible future outcomes.
How it connects
Connects reward to Markov decision processes, dynamic programming, Bellman equations, reinforcement learning, and Q-learning.

Also explained in

Compare how 16 books explain reward signal →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.