liveBook · Manning

Dense reward

As explained in Applied Reinforcement Learning

A stepwise reward is feedback assigned at individual decision steps rather than only at the end of an episode or process.

In a multi-step business task, a dense reward gives an agent feedback at each decision rather than waiting until the episode ends. It can turn KPIs such as delivery reliability or profit into stepwise signals that guide learning. section 2.6.1

Useful stepwise signals can make a distant goal more actionable:

  • For delivery routing, penalize each unnecessary turn instead of revealing fuel use only after the route. section 2.6.1
  • For retail inventory, score daily demand met and penalize unsold stock and stockouts. section 2.6.1
  • When campaign results arrive late, use intermediate measures such as click-through rates or engagement time as directional feedback. section 2.6.1

A frequent reward can still steer behavior in the wrong direction if it measures a proxy. Rewarding machine runtime may create unneeded inventory; rewarding clicks may favor clickbait over sales and user satisfaction. Test and monitor the behavior the reward produces. section 2.6.6

How Applied Reinforcement Learning covers it

How deep it goes
Explains reward timing with routing and inventory examples, then warns how poorly chosen proxies can distort behavior.
Examples
Uses delivery routing, fashion inventory, delayed marketing measures, machine utilization, and recommendation clicks.
What you can do after
Apply dense rewards by choosing intermediate business signals and monitoring for behavior that exploits proxy metrics.
How it connects
Connects dense rewards with sparse rewards, episodic problems, KPI design, and reward engineering.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.