Concept drift occurs when changes in a domain alter the relationships between inputs and outcomes, reducing a previously trained model’s applicability. Performance can also deteriorate when data noise changes; interface changes may produce related feature drift. section 10.2.2
Manage concept drift as an operational model-risk concern:
- Review whether the model’s causal relationships still hold as customer behavior and external conditions change. section 10.2.2
- Monitor changing noise levels and noise types, which can increase errors that were previously unimportant. section 10.2.2
- Establish behavioral assertions and monitor the model to detect drift caused by changing technical interfaces or infrastructure. section 10.2.2
Work-from-home disruption
A model learns that customers often buy smart trousers and shirts together. When a pandemic moves work home, demand for smart trousers disappears; recommendations for jeans or jogging trousers become more effective until the model is adapted. section 10.2.2
