Concept drift occurs when the relationship between inputs and correct outputs changes. A model may give outdated answers even when the questions themselves have not changed. section 9.2
When a correct answer goes stale
A support model directs “password reset” questions to an Active Directory self-service portal. After the organization replaces that portal with a cloud SSO provider, the same answers are wrong. section 9.2
Do not confuse this with data drift, where the input distribution changes, or prompt drift, where users change how they phrase requests. Vocabulary-distance detectors can flag data or prompt drift, but canary prompts are needed to catch answers made stale by concept drift. section 9.2 section 9.2.3
Use a quarterly review to decide whether conditions have changed:
- Sample recent production questions and compare their vocabulary with the training set. section 9.6.4
- Ask support leads what changed in products, policies, processes, or terminology. section 9.6.4
- Assess retraining when metrics indicate degradation; do not retrain automatically on a schedule. section 9.6.4
Figure 9.3 Responding to a drift signal: a red drift score triggers a canary run before any retrain decision.

A red drift score triggers a canary run before any retraining decision. section 9.2.3







