A model can keep receiving familiar-looking inputs and still make the wrong predictions. Concept drift occurs when the relationship between input features and expected outcomes changes, making learned patterns unreliable or obsolete. Unlike data drift, the distribution of incoming inputs need not change. section 14.1.3 section 14.4.2
When a tax rule changes
Suppose a marketing model selects people likely to need tax-service assistance while keeping a campaign within budget. If the threshold for the highest UK tax bracket falls from £150,000 to £125,000, the same income can imply a different need. The model’s old targeting pattern may no longer serve the campaign. section 14.1.3
Figure 14.7 With a concept drift, identical inputs may lead to new expected outputs; with a data drift, incoming data changes while the model is not adapted.
Compare what changes: with concept drift, identical inputs may have new expected outputs; with data drift, the incoming inputs change. section 14.4
Consider both the pace of change and its effect on performance:
- Gradual drift can follow evolving consumer behavior, economic conditions, or equipment wear. Performance may decline over time. section 14.4.2
- Sudden drift can follow a drastic external event. The COVID-19 pandemic, for example, changed mobility and shopping patterns almost overnight. section 14.4.2
- Recurring drift is another identified pattern; do not assume every change is permanent. section 14.4.2
In production, track performance metrics against historical baselines. A decline may signal model decay, but it does not by itself establish the cause: check for data-quality problems and investigate whether the feature–outcome relationship has genuinely changed. Visualizing those relationships and consulting domain experts can help guide the response. section 14.4 section 14.4.4
If the relationship has changed, retraining may restore accuracy; a model may instead need to be rebuilt. Test older data with different retraining frequencies to estimate how quickly performance ages. When retraining for concept drift, consider whether to use all available data, give newer data more weight, or drop older data when enough new data exists. Do not treat a working training pipeline as a reason to retrain before diagnosing the problem. section 14.4.2 section 14.4.4 section 14.6
