liveBook · Manning

Reward engineering

As explained in Applied Reinforcement Learning

The process of specifying how outcomes or behaviors are evaluated and mapped to reward feedback.

Reward engineering translates business goals and rules into scalar feedback an agent can optimize. Since the agent responds to the number it receives, the reward must represent both desired outcomes and acceptable boundaries. section 2.6

Design the signal around the decisions the agent must make:

  • When a task unfolds over time, consider stepwise rewards that give feedback at each decision rather than only at the end. Intermediate measures can approximate outcomes that take longer to observe. section 2.6.1
  • Balance multiple goals and constraints so that a reward for one objective does not undermine the others. section 2.6
  • Test and monitor the behavior the reward produces; an agent may exploit a loophole or optimize the wrong trade-off while still earning a high score. section 2.6.6 section 12.9.2
reward = ($0.02 * click_count) + ($0.10 * revenue) - ($0.05 * bounce_rate).
 

A weighted score combines clicks, revenue, and bounce rate, illustrating how a reward can encode competing signals. section 2.6.6

A risk penalty that overwhelms returns

In portfolio management, risk penalties set too high led an agent to move almost everything into cash and stop trading. The reward looked reasonable, but the learned behavior was useless; checking what the agent actually did revealed the problem. section 12.9.2

How Applied Reinforcement Learning covers it

How deep it goes
Explains stepwise feedback, competing objectives, and reward loopholes through business examples, without detailing a full implementation process.
Examples
Uses delivery routing, fashion inventory, clickbait recommendations, and portfolio risk penalties to illustrate reward choices.
What you can do after
Diagnose reward hacking by testing the behavior an agent learns, not just the score it receives.
How it connects
Connects reward design to reinforcement learning, Markov decision processes, and constraint handling.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.