liveBook · Manning

Importance weight

As explained in AI Model Evaluation

An importance weight is a factor that reweights an observation to account for a difference between the distribution that generated it and a target distribution.

An importance weight adjusts an observed outcome for a difference between the policy that generated it and a target policy. In off-policy evaluation, the weight is the probability that the new policy would take the observed action divided by the probability that the logged policy took it. Weighted outcomes help estimate how the new policy would perform using existing data. section 5.4.2

Reweighting a recommendation

If the logged policy showed an item 10% of the time and the new policy would show it 30% of the time, the observed outcome gets a weight of 3. The adjusted outcomes can then contribute to an estimate of expected reward or another policy metric. section 5.4.2

Listing 5.2 Calculating Off Policy Evaluation with multiple metrics
def evaluate_policy(self, metric='engagement_score'):
        weights = self.calculate_importance_weights()
        
        if metric == 'click_rate':
            rewards = self.logged_data['clicked'].astype(float)
        elif metric == 'watch_time':
            rewards = self.logged_data['watch_time']
        else:  # engagement_score
            rewards = (self.logged_data['clicked'].astype(float) + self.logged_data['watch_time'] / 100)
        
        new_policy_value = np.sum(weights * rewards) / np.sum(weights) if np.sum(weights) > 0 else 0
        original_policy_value = np.mean(rewards)
        
        return {
            'original_policy_value': original_policy_value,
            'new_policy_value': new_policy_value,
            'improvement': new_policy_value - original_policy_value,
            'relative_improvement': ((new_policy_value / original_policy_value - 1) * 100 if original_policy_value > 0 else 0),
            'effective_sample_size': (np.sum(weights) ** 2) / np.sum(weights ** 2) if np.sum(weights) > 0 else 0
        }
 

The evaluation code applies importance weights to click rate, watch time, or an engagement score, then divides the weighted reward sum by the sum of weights. section 5.4.2

Reliable weighting depends on accurate logged probabilities and sufficient overlap between policies. Limited exploration, a large action space, or policies that differ substantially can leave too little useful evidence. Inspect the weight distribution, effective sample size, and confidence intervals; if clipping large weights or changing the time window makes the estimate swing wildly, it may not support a high-confidence decision. section 5.4.2 section 5.3.2

How AI Model Evaluation covers it

How deep it goes
Defines the policy probability ratio and demonstrates weighted evaluation, with practical checks for unstable estimates.
Examples
Uses a 10%-to-30% recommendation probability change and a movie evaluation with click and watch-time rewards.
What you can do after
Evaluate whether weighted offline policy estimates are sufficiently stable for a decision.
How it connects
Connects importance weights to inverse propensity weighting, off-policy evaluation, and effective sample size.

Also explained in

Compare how 5 books explain importance weight →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.