liveBook · Manning

Proxy reward function

As explained in Applied Reinforcement Learning

A proxy reward is an observable signal that serves as an approximation for a later or less observable desired outcome.

A proxy reward is an observable early signal used to approximate a later or less observable outcome. It helps learning continue when the true result arrives slowly or remains censored. section 5.6

Marketing feedback

When a purchase may occur after several visits, use an email open or add-to-cart event as an interim signal, or buffer the pending outcome until enough information arrives for a fair update. section 5.6

How Applied Reinforcement Learning covers it

How deep it goes
Defines the role of proxy signals and contrasts them with buffering pending outcomes, without deriving an update rule.
Examples
Uses email opens and add-to-cart events as early signals for purchases after multiple visits.
What it assumes
Assumes familiarity with bandit decisions and reward feedback, while treating delayed and censored outcomes practically.
What you can do after
Apply it via early behavioral signals or buffered outcomes when direct feedback cannot update learning immediately.
How it connects
Connects proxy rewards to delayed rewards, censored rewards, and contextual bandits.

Also explained in

Compare how 3 books explain proxy reward function →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.