liveBook · Manning

Probability theory

As explained in AI Model Evaluation

Probability is a numerical measure from zero to one representing the likelihood of an event or choice.

Probability measures the likelihood of an event or choice on a scale from zero to one. In a recommendation system, action probabilities describe how likely a policy is to select each available item. Recording them helps teams evaluate a new policy without exposing users to the change. section 5.4.2 section 5.5.2

To evaluate a new policy, log the decision context, candidate actions, selected action, original policy probabilities, and observed outcome. Calculate the new policy’s probability for the logged action, then use inverse propensity weighting to adjust the outcome by the ratio of the new probability to the original probability. Aggregate the weighted outcomes to estimate policy value. section 5.11 section 5.8.1

Movie recommendation

For each session, record the recommended movie, alternatives and their original probabilities, viewing context, and outcomes such as clicks or watches. Apply the new policy to calculate its probabilities, then weight the observed outcomes to estimate its overall value. section 5.8.1

Listing 5.1 Calculating New Policy Probabilities
def new_policy_probabilities(self, user_id, available_movie_ids):
        """Quality-focused policy (vs popularity-based logged policy)"""
        user_prefs = self.users_catalog.iloc[user_id]['preferred_genres']
        available_movies = self.movies_catalog[self.movies_catalog['movie_id'].isin(available_movie_ids)]
        
        scores = []
        for _, movie in available_movies.iterrows():
            score = movie['rating']  # Focus on quality
            if movie['genre'] in user_prefs:
                score *= 2.0  # Strong preference boost
            scores.append(score)
        
        scores = np.array(scores)
        if len(scores) == 0:
            return np.array([])
        
        exp_scores = np.exp(scores - np.max(scores))
        return exp_scores / np.sum(exp_scores)
 

See how a quality-focused movie policy scores available titles, boosts preferred genres, and normalizes the scores into probabilities. section 5.4.2

These estimates are not perfect replays of what would have happened. Accurate probabilities and sufficient overlap between the original and new policies matter; inspect weight distributions, effective sample size, confidence intervals, and segment-level results before trusting a comparison. section 5.11

How AI Model Evaluation covers it

How deep it goes
Explains probability logging, policy-weighted outcome aggregation, and checks for unstable estimates.
Examples
Calculates movie recommendation probabilities from ratings and genre preferences, then evaluates logged clicks or watches.
What you can do after
Evaluate a new policy using logged probabilities and weighted outcomes, while checking whether the estimate is trustworthy.
How it connects
Connects probability to causal inference, off-policy evaluation, policy value, and inverse propensity weighting.

Also explained in

Compare how 22 books explain probability theory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.