Model drift is a change over time in a model or evaluation judge that can undermine evaluation while outputs remain plausible. A hosted judge may change behavior behind a stable-looking API identifier even when its prompt, rubric, and pipeline stay unchanged; score changes may then reflect the judge, not the product. section 10.6.1
Separate the possible causes and use evaluations suited to the setting:
- For hosted judges, use canary evaluation to help distinguish a drifting judge from a changing product. section 10.6.1
- For internal models, offline evaluation can track performance over time. Historical support tickets can be compared with agent labels using accuracy, critical-ticket recall, or resolution-time proxies. section 1.3.2
- For internal support agents, replay historical tickets and assess outputs against known resolutions or human-written responses. section 1.3.2

















