A policy is a decision rule that takes context and chooses an action, or assigns probabilities to possible actions. It can be a recommendation algorithm, ranking model, tool-routing strategy, or business rule. Policy value is its expected performance on a chosen outcome, such as click-through rate or watch time. section 5.4.1
Before exposing users to a new policy, off-policy evaluation estimates its value using decisions made by an existing policy. This can help compare a candidate with a baseline, but the result is an estimate, not a replay of what would have happened. section 5.11 section 5.4.2
A practical evaluation follows the decision from logging to comparison:
- Log the decision context, candidate set, selected action, its probability, policy version, and observed outcome. Capture factors that mattered at decision time. section 5.3 section 5.2.2
- Replay each logged context through the candidate policy and calculate its probability of taking the action actually logged. Weight the observed outcome by the candidate probability divided by the logged policy’s probability. section 5.4.1
- Aggregate weighted outcomes for the chosen metric and compare the estimate with the baseline. Inspect confidence intervals, effective sample size, weight distributions, and segment-level results. section 5.8.1 section 5.11
Reliable estimates require accurate action probabilities, sufficient overlap between policies, and relevant logged context. A large dataset is not enough if the old policy rarely explored actions the new policy favors. Unstable weights or estimates that swing across segments or time windows are warning signs. Use the estimate to decide whether a policy warrants an online A/B test, not as proof of user impact. section 5.2.2 section 5.3.2
Evaluating a movie recommendation policy
A candidate policy can score available movies by rating, boost scores for preferred genres, and convert the scores into action probabilities. Logged recommendations, alternatives, session context, and outcomes support a weighted comparison with the original policy. In one illustrative outcome, the candidate estimated 15% higher average viewing time and 10% higher average ratings, with a stronger viewing-time increase on weekends. section 5.4.2 section 5.8.1 section 5.8.3
Listing 5.1 Calculating New Policy Probabilities
def new_policy_probabilities(self, user_id, available_movie_ids):
"""Quality-focused policy (vs popularity-based logged policy)"""
user_prefs = self.users_catalog.iloc[user_id]['preferred_genres']
available_movies = self.movies_catalog[self.movies_catalog['movie_id'].isin(available_movie_ids)]
scores = []
for _, movie in available_movies.iterrows():
score = movie['rating'] # Focus on quality
if movie['genre'] in user_prefs:
score *= 2.0 # Strong preference boost
scores.append(score)
scores = np.array(scores)
if len(scores) == 0:
return np.array([])
exp_scores = np.exp(scores - np.max(scores))
return exp_scores / np.sum(exp_scores)
Inspect how movie ratings and genre preferences become probabilities over the available candidates. section 5.4.2




















