Probability measures the likelihood of an event or choice on a scale from zero to one. In a recommendation system, action probabilities describe how likely a policy is to select each available item. Recording them helps teams evaluate a new policy without exposing users to the change. section 5.4.2 section 5.5.2
To evaluate a new policy, log the decision context, candidate actions, selected action, original policy probabilities, and observed outcome. Calculate the new policy’s probability for the logged action, then use inverse propensity weighting to adjust the outcome by the ratio of the new probability to the original probability. Aggregate the weighted outcomes to estimate policy value. section 5.11 section 5.8.1
Movie recommendation
For each session, record the recommended movie, alternatives and their original probabilities, viewing context, and outcomes such as clicks or watches. Apply the new policy to calculate its probabilities, then weight the observed outcomes to estimate its overall value. section 5.8.1
Listing 5.1 Calculating New Policy Probabilities
def new_policy_probabilities(self, user_id, available_movie_ids):
"""Quality-focused policy (vs popularity-based logged policy)"""
user_prefs = self.users_catalog.iloc[user_id]['preferred_genres']
available_movies = self.movies_catalog[self.movies_catalog['movie_id'].isin(available_movie_ids)]
scores = []
for _, movie in available_movies.iterrows():
score = movie['rating'] # Focus on quality
if movie['genre'] in user_prefs:
score *= 2.0 # Strong preference boost
scores.append(score)
scores = np.array(scores)
if len(scores) == 0:
return np.array([])
exp_scores = np.exp(scores - np.max(scores))
return exp_scores / np.sum(exp_scores)
See how a quality-focused movie policy scores available titles, boosts preferred genres, and normalizes the scores into probabilities. section 5.4.2
These estimates are not perfect replays of what would have happened. Accurate probabilities and sufficient overlap between the original and new policies matter; inspect weight distributions, effective sample size, confidence intervals, and segment-level results before trusting a comparison. section 5.11





















