Causal inference estimates what caused an outcome rather than treating correlation as proof. It helps determine whether an intervention—such as a recommendation, search ranking, or product change—actually produced a user or business effect. section 5.1
The central question is counterfactual: what would have happened if a different decision had been made? Engineering systems cannot replay that alternate reality directly, but logged decisions and suitable methods can estimate it before exposing a change to users. section 5.1 section 5.3.1
A practical causal-analysis workflow connects decisions, outcomes, and evidence:
- Preserve the decision context, candidate actions, selected action, action probability or score, policy version, and observed outcome. section 5.11
- Estimate the value of a different policy using off-policy evaluation; inverse propensity weighting reweights observed outcomes according to how likely the new policy was to select each logged action. section 5.11
- Check overlap, propensity accuracy, weight stability, effective sample size, confidence intervals, and segment-level results before trusting the estimate. section 5.11 section 5.3
Recommendation policy comparison
An e-commerce site logs each user's context, the product shown, the serving policy's selection probability, and whether the user clicked or purchased. Those records can estimate how another recommendation policy might perform before an online test. The estimate remains conditional: if the new policy rarely chooses actions taken by the logged policy, the available evidence may be too weak for a reliable comparison. section 5.3 section 5.11


