liveBook · Manning

Continuous improvement

As explained in AI Model Evaluation

A dynamic process in which a model's predictions influence user behavior and that behavior changes the data observed by the model later.

A model feedback loop occurs when predictions influence user behavior, and that behavior changes the data the model later uses. The cycle can reinforce existing preferences or errors, so higher engagement does not necessarily mean a better user experience. section 1.8.1 section 8.2

Figure 8.1 illustrates feedback loop in AI model evaluations. The model influences user behavior, which generates logged data that feeds back into retraining. Pitfalls such as exposure bias, reinforced errors, and satisfaction masking emerge along the loop, making evaluation an evolving target rather than a clean snapshot.

Follow the cycle from model outputs through user behavior and logged data to retraining, including the evaluation pitfalls that can emerge. section 8.2

A news feed reinforces a topic

A feed recommends more political content. The user engages with it, so the model recommends even more political content later. Engagement may grow partly because the user saw more of that topic, not because each recommendation became more relevant. section 1.8.1 section 8.2

This behavioral loop differs from a deliberate development loop: a team can use diagnostics to identify a problem, change a model, and rerun evaluations. Both involve iteration, but the behavioral loop also changes the data used to judge future performance. section 3.4.6 section 8.2

Design evaluations to reveal feedback-driven effects:

  • Use offline evaluations to screen changes, but recognize that static historical data cannot capture the full effect of recommendations on future behavior. Supplement them with simulations or online tests aimed at long-term effects. section 1.2 section 1.8.1
  • Measure more than immediate clicks or dwell time. A loop can improve short-term engagement while satisfaction, retention, or diversity declines. section 8.2
  • Add exploration through randomized exposure or a small multi-armed bandit traffic allocation. Log decision propensities to support counterfactual and off-policy analysis. section 8.2.1
  • Keep a stable baseline holdout to help distinguish model improvement from feedback-driven distribution shift. Balance its value against maintenance costs and the user experience of remaining on an older model. section 8.2.1

How AI Model Evaluation covers it

How deep it goes
Explains the feedback cycle, evaluation pitfalls, and safeguards, including exploration and holdout trade-offs.
Examples
Uses a political news feed to illustrate how engagement can reinforce recommendations over time.
What you can do after
Evaluate and understand feedback-driven metric changes using exploration, long-term holdouts, and measures beyond short-term engagement.
How it connects
Connects recommendation systems with exposure bias, counterfactual evaluation, multi-armed bandits, and distribution shift.

Also explained in

Compare how 43 books explain continuous improvement →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.