liveBook · Manning

Action-outcome pair

As explained in AI Model Evaluation

An action-outcome pair records a system action together with the resulting observed outcome.

An action-outcome pair records an action taken by a system together with the resulting observed outcome. It turns historical interaction data into structured evidence for comparing predictions with what happened. section 2.2.1

Use action-outcome pairs to:

  • Represent interactions such as recommendations followed by clicks, watches, purchases, saves, or ratings. section 2.2.1
  • Support replay and counterfactual evaluation when combined with propensity scores and contextual metadata. section 2.2.1
  • Interpret outcomes cautiously because observed behavior can reflect placement or exposure rather than true preference. section 2.2.1

Movie recommendation

A recommendation system displays a movie, and the log records the resulting click, watch, save, rating, or purchase. That action-outcome pair can help compare predictions with observed behavior, while recognizing that prominent placement may influence the result. section 2.2.1

How AI Model Evaluation covers it

How deep it goes
Defines the pair and its evaluation role, but does not detail logging schemas or estimation procedures.
Examples
Uses movie recommendations paired with clicks, watches, purchases, saves, or ratings.
What you can do after
Evaluate predictions against observed outcomes and support counterfactual metrics when paired with propensity scores.
How it connects
Connects to offline evaluation, propensity scores, contextual metadata, and counterfactual metrics.

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.