An investment can pay off or fail, and the choice to invest does not guarantee either result. A reward signal assigns a value to what happens so an agent can compare actions. In a simple investment model, the utility or reward function is U(Y): a $1,000 investment yields utility of $99,000 if it returns $100,000, and −$1,000 if it returns nothing. Loss or cost reverses the sign: L(Y) = −U(Y). section 12.1.1
A reward value alone does not tell the agent which action to take. Other causes, such as economic conditions, can change the outcome or directly affect utility. To choose between actions, compare the reward distributions under the actions being considered—not merely the outcomes observed among people who previously chose them. One decision rule selects the action x with the greatest expected utility E[U(Yₓ)], where Yₓ denotes the outcome under that action. A minimax rule instead limits worst-case loss; a softmax rule allows less-than-optimal choices. section 12.1.2 section 12.2.2
The timing and structure of feedback change how the signal is used:
- With bandit feedback, an action produces an outcome and its utility immediately; trials are independent across timesteps. A policy can nevertheless use past actions and reward outcomes to select the next action. section 12.3.1 section 12.4.1
- In sequential care decisions, a medication policy considers earlier pain reports and doses. Its utility accounts for pain, overdose risk, and addiction risk rather than only the latest pain report. section 12.4.1
- In a Markov decision process, rewards sit alongside states, actions, and transitions. A grid-world agent receives 100 for reaching a goal square and −100 for entering a square it should avoid. section 12.5.1
Figure 12.27 A simple grid world
The grid makes positive and negative rewards concrete: the agent must navigate around an obstacle toward the rewarded goal while avoiding the penalized square. section 12.5.1
Implementation depends on the decision being modeled. Represent reward as a utility node, specify what determines its value, and evaluate candidate actions against the resulting utility distribution. In a causal model, intervening on an action matters: an observed association between an action and a high reward need not be the reward that choosing that action would produce. section 12.1.1 section 12.1.2 section 12.2.6
