Model degradation is the decline in a deployed model’s predictive or operational performance over time. As the target population’s statistical distribution changes, predictions become less accurate, making degradation a production monitoring concern. section 8.2.1
Track degradation through serving and performance metadata:
- Record model execution data, including response latency and prediction miss rate, for later performance analysis. section 8.2.1
- Track evaluation metrics for drift detection, including concept drift and performance drift. section 8.2.1
- Preserve model lineage and training metrics so teams can compare models and reproduce how each model was produced. section 8.2.1
Changing language
New popular slogans can change the language distribution encountered by a voice-recognition model, causing its predictions to become less accurate after deployment. section 8.2.1
