liveBook · Manning

Variance inflation

As explained in AI Model Evaluation

An increase in the variance of an estimate or comparison caused by additional sources of variation among the observations or groups.

Variance inflation occurs when feedback loops make treatment and control populations increasingly different, adding behavior-driven variation to an A/B comparison. In a movie recommender, narrowing treatment recommendations toward blockbusters can make treatment users’ watch histories more homogeneous while control users remain diverse. The comparison then reflects both the model change and the behaviors it triggered. section 8.2

Manage variance inflation with evaluation designs that preserve comparability:

  • Use stratified analysis across stable cohorts such as tenure, geography, viewing intensity, or genre preference. section 8.7.4
  • Reserve exploration traffic for randomized exposure, creating an unbiased counterfactual baseline. section 8.7.4
  • Maintain a long-term holdback on an invariant baseline model to distinguish genuine improvement from feedback-induced drift. section 8.7.4
  • Prefer shorter, well-designed tests or persistent holdouts when longer experiments allow populations to diverge. section 8.2

Movie recommendation experiment

A treatment recommender promotes a narrow set of blockbuster titles while control serves a more diverse slate. Initially, users are comparable; weeks later, treatment users binge similar titles and develop homogeneous histories. The resulting variance makes the measured difference less attributable to the treatment alone. section 8.2

Figure 8.1 illustrates feedback loop in AI model evaluations. The model influences user behavior, which generates logged data that feeds back into retraining. Pitfalls such as exposure bias, reinforced errors, and satisfaction masking emerge along the loop, making evaluation an evolving target rather than a clean snapshot.

Trace how model recommendations alter user behavior, logged data, and later evaluation, creating an evolving target rather than a clean snapshot. section 8.2

How AI Model Evaluation covers it

How deep it goes
Explains the causal mechanism and supplies stratification, exploration, and holdback tactics, without formal variance calculations.
Examples
Uses a movie recommender that promotes blockbusters and compares increasingly homogeneous treatment histories with diverse control histories.
What it assumes
Assumes treatment-control experiments and feedback loops, while establishing how model-driven behavior changes undermine comparability.
What you can do after
Diagnose variance inflation with cohort slices, randomized exploration, shorter tests, and invariant baseline holdouts.
How it connects
Connects feedback loops, A/B testing, offline evaluation, counterfactual logging bias, exploration traffic, and long-term holdouts.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.