A bank may deploy a fraud detector that performs well today, then find that it misses new fraud tactics months later. Concept drift is a change in the relationship between transaction information and the fraud outcome the model is meant to predict. Fraudsters adapt to defenses, new technologies, economic events, and financial products, so patterns learned from historical cases may lose predictive power. section 7.3 section 8.1.3
Concept drift is distinct from data drift: concept drift concerns the nature of fraud, while data drift concerns the characteristics of incoming data. The Population Stability Index (PSI) is given as a possible measure of data drift, not as a direct measure of changing fraud tactics. A falling fraud detection rate can signal a problem, but performance and operational measures need to be considered together. section 7.4.4
Treat drift as an ongoing fraud-operations concern:
- Monitor precision, recall, AUC, false positive ratio, fraud detection rate, fraud value detected versus missed, and cost per investigation. These measures expose both missed fraud and customer or investigation costs. section 7.4.4 section 7.3
- Capture investigation outcomes, including confirmed fraud and legitimate but unusual activity. Feed them back into model retraining, rule and threshold refinement, and watchlist updates. section 7.4.4
- Validate models regularly and retrain on fresh legitimate behavior and newly identified fraud cases. Review effectiveness, fairness, and regulatory compliance as defenses change. section 8.1.3 section 7.4.4
A new fraud tactic
A supervised detector trained on past transactions may fail to recognize fraud exploiting a newly launched financial product. Newly confirmed cases can inform retraining, while monitoring checks whether detection improves without an unacceptable rise in legitimate transactions being blocked. section 7.3 section 8.1.3 section 7.4.4
Figure 7.3 Core characteristics that distinguish financial fraud from other AI challenges. The central icon, a scroll representing a financial record ($) under scrutiny (eye), symbolizes the core challenge of identifying a fraudulent financial event. This task is uniquely difficult due to a set of interconnected factors: extreme class imbalance (Sparsity of events), the need for explainability, complex multisource data, organized and adaptive attackers, the rapid evolution of fraud tactics, and the high costs of both false negatives and false positives.
The diagram places rapidly evolving tactics alongside adaptive attackers, rare fraud events, explainability needs, and the costs of both missed fraud and false alarms. section 7.3
