8 Building AI fraud detectors: from supervised scoring to anomaly identification

 

This chapter covers

  • Establishing core principles for AI-driven fraud detection
  • Preparing specialized data for fraud analytics, addressing imbalance and privacy
  • Implementing supervised learning for known fraud patterns
  • Evaluating supervised models with business-relevant and strategic metrics
  • Applying unsupervised learning to detect novel anomalies

The shadowy world of financial fraud is a high-stakes arena where innovation is a weapon wielded by both perpetrators and defenders. As we explored in Chapter 7, the adversaries are nimble, constantly adapting their tactics to find the faintest cracks in institutional defenses. To counter this ever-evolving threat, understanding fraud isn’t enough—we must build systems that learn, adapt, and respond swiftly. This chapter transitions from understanding fraud to building AI systems that fight it. We embark on a practical journey, constructing AI models to unmask illicit activities in vast financial data streams.

At its heart, fighting financial fraud with AI revolves around three fundamental quests. This chapter focuses on the first two: recognizing the known, which uses supervised learning to spot familiar fraud patterns, and finding the unknown, which uses unsupervised anomaly detection to identify novel threats. The third quest, unveiling hidden connections through graph analytics, will be covered in Chapter 9.

8.1 Recognizing the known: the power and nuances of supervised learning in fraud detection

8.1.1 The cornerstone: acquiring, managing, and wrestling with labeled data

8.1.2 Beyond binary labels: achieving contextual understanding in fraud classification

8.1.3 Practical realities: the ongoing vigilance in supervised fraud modeling

8.2 Data preparation and initial exploration for supervised modeling

8.2.1 Loading and understanding our dataset: the Kaggle credit card fraud data

8.2.2 Essential preprocessing: scaling features and splitting the data

8.3 Supervised fraud classification in action: training and tuning a LightGBM model

8.3.1 Defining the modeling pipeline and hyperparameter search space

8.3.2 Training the model with randomized hyperparameter search

8.3.3 Evaluating the tuned model: comprehensive metrics on the test set

8.3.4 Strategic insight: analyzing performance at top-N% prediction thresholds

8.4 Finding the unknown: unsupervised anomaly detection with autoencoders

8.4.1 Autoencoders: Learning Normality to Detect Deviance

8.4.2 Preparing data for the autoencoder

8.4.3 Building and training the autoencoder model

8.4.4 Calculating and examining reconstruction errors

8.4.5 Visualizing reconstruction error distributions

8.4.6 Setting practical thresholds in an unsupervised world

8.4.7 Deploying real-time fraud scoring models: practical considerations

8.5 Summary