A correlation matrix organizes pairwise relationships between metrics in a table. Rows can represent offline metrics, columns online metrics, and each cell the strength of their relationship. section 7.6.3
Use the matrix to turn evaluation results into model-selection guidance:
- Compare relationships such as precision@10 with online CTR lift or diversity index with online session length. section 7.6.3
- Start with Pearson’s r when exact change magnitude matters, then check Spearman’s rho for robustness against outliers or non-linear patterns. section 7.6.3
- Update the matrix after every experiment and increase the model-selection weight of metrics that consistently predict meaningful online outcomes. section 7.6.3
Figure 7.2 illustrates the offline to online correlation feedback loop.
Follow the offline-to-online correlation feedback loop from measured relationships to revised model-selection priorities. section 7.6.3





