8 Building AI fraud detectors: from supervised scoring to anomaly identification
This chapter covers
- Establishing core principles for AI-driven fraud detection
- Preparing specialized data for fraud analytics, addressing imbalance and privacy
- Implementing supervised learning for known fraud patterns
- Evaluating supervised models with business-relevant and strategic metrics
- Applying unsupervised learning to detect novel anomalies
In Chapter 7, we established the theoretical and architectural foundations of financial fraud detection. Now, we transition from concept to code, constructing the actual machine learning models that operate within the Modeling Layer to unmask illicit activities in financial data streams.
At its core, AI-driven fraud detection is divided into two primary analytical domains: evaluating isolated transactions and analyzing interconnected networks. This chapter focuses entirely on single-transaction detection. We will tackle this through two distinct machine learning tasks: using supervised learning to recognize known fraud patterns, and using unsupervised anomaly detection to flag novel, previously unseen threats. The second domain—unveiling colluding entities through complex network and graph analytics—will be the focus of Chapter 9.
8.1 Recognizing the known: the power and nuances of supervised learning in fraud detection
8.1.1 The cornerstone: acquiring, managing, and wrestling with labeled data
8.1.2 Beyond binary labels: achieving contextual understanding in fraud classification
8.1.3 Practical realities: the ongoing vigilance in supervised fraud modeling
8.2 Data preparation and initial exploration for supervised modeling
8.2.1 Loading and understanding our dataset: the Kaggle credit card fraud data
8.2.2 Essential preprocessing: scaling features and splitting the data
8.3 Supervised fraud classification in action: training and tuning a LightGBM model
8.3.1 Defining the modeling pipeline and hyperparameter search space
8.3.2 Training the model with randomized hyperparameter search
8.3.3 Evaluating the tuned model: comprehensive metrics on the test set
8.3.4 Strategic insight: analyzing performance at top-N% prediction thresholds
8.4 Finding the unknown: unsupervised anomaly detection with autoencoders
8.4.1 Autoencoders: Learning Normality to Detect Deviance
8.4.2 Preparing data for the autoencoder
8.4.3 Building and training the autoencoder model
8.4.4 Calculating and examining reconstruction errors
8.4.5 Visualizing reconstruction error distributions
8.4.6 Setting practical thresholds in an unsupervised world
8.4.7 Deploying real-time fraud scoring models: practical considerations
8.5 Summary