A deployed model does not have to stay fixed. It may be updated as data arrives or retrained on a schedule, such as every week using the latest batch. Either way, its weights can change, so the same input may produce a different output. That change can be the point of retraining—but it also needs to be justified and checked. section 14.1.2
When data drift appears, do not treat the retrain button as the first diagnostic step:
- Watch performance measures such as accuracy, mean error, or fraud rates. A drifted input distribution does not, by itself, establish that the model needs replacing. section 14.4.4
- Check for data-quality faults, including entry errors, schema changes, and problems with upstream models. Fix those separately from genuine shifts in the patterns being modeled. section 14.4.4
- If the shift is genuine, inspect distributions of drifting features and consult domain experts about possible causes. Consider additional data or features; in some cases, retrain from scratch on updated data. section 14.4.4
An antifraud release
New data may improve an antifraud model, but repeated retraining should not casually label the same user fraudulent today, legitimate tomorrow, and fraudulent again next week. Compare outputs on unchanged inputs across releases with consistency metrics, alongside ordinary performance metrics. The aim is not to forbid change; it is to spot changes that disrupt downstream systems or users. section 5.2.1 section 14.1.2
