Expected reward estimates the total benefit a new policy could produce, such as revenue or engagement. In off-policy evaluation, it turns logged decisions and outcomes into a counterfactual policy comparison. section 5.4.2
Estimate expected reward by:
- Defining the policy and the outcome that represents value. section 5.4.2
- Applying the policy to logged contexts and comparing its action probabilities with those recorded by the original policy. section 5.4.2
- Using inverse propensity weighting to correct for actions appearing frequently because the original policy preferred them. section 5.4.2
Movie recommendation policy
A quality-focused recommender can be evaluated against a popularity-based logged policy using click rate, watch time, or a combined engagement score. The result includes original policy value, new policy value, and improvement.
