Correlation measures how strongly two variables move together. A value near +1 indicates that they rise and fall together, a value near −1 indicates opposite movement, and a value near 0 indicates little relationship. Correlation does not establish that one variable causes another. section 7.6.3 section 5.1
In model evaluation, correlation connects offline scores with online experiment outcomes. A useful process maintains a rolling dataset containing variant identifiers, experiment dates, offline metrics, online metric changes, traffic allocation, experiment length, product domain, and model maturity. section 7.6.3
Use the relationship to test whether offline improvements predict product results:
- Record offline measures such as precision@K, recall@K, NDCG, diversity, fairness gaps, and latency. section 7.6.3
- Record corresponding online changes such as click-through rate, conversion, retention, engagement depth, or watch time. section 7.6.3
- Compare each offline metric with each online metric over repeated experiments rather than assuming that one offline score predicts success. section 7.6.3
- Update the relationships over time because metric definitions, logging schemas, and product conditions can change. section 7.6.3 section 7.8.3
Movie recommendation evaluation
Suppose precision@10 improves for a recommendation variant. Correlation tracking asks whether experiments also show higher click-through rate, watch time, or retention. A positive relationship can support model-selection decisions, but it cannot prove that the recommendation caused the outcome; causal inference is needed for that question. section 7.6.3 section 5.1
Figure 7.2 illustrates the offline to online correlation feedback loop.
Trace the feedback loop linking offline evaluation results, online experiments, historical correlations, and later model-selection decisions. section 7.6.3





















