Population stability index (PSI) measures how a prediction population differs from a reference population. In production monitoring, it can serve as an early warning for prediction drift before delayed truth data reveals a model-quality problem. section 14.5.1
A practical monitoring setup treats PSI as one candidate signal rather than an unquestioned decision rule:
- Track next-day, next-week, and next-month predictions for drift. section 14.5.1
- Test an alert threshold of PSI greater than 0.2 alongside Wasserstein distance greater than 0.1. section 14.5.1
- Adjust the reference data for known growth before comparing periods, then refine the approach through historical experimentation. section 14.5.1
- Use delayed ground-truth metrics to assess model relevance, while retaining drift alerts to detect change earlier. section 14.5.1
