A fraud team needs a way to spot unusual card transactions, including patterns absent from past fraud cases. An autoencoder trained only on legitimate transactions learns to reproduce familiar inputs. Reconstruction loss measures the difference between a transaction’s original features and the model’s reconstructed features. A large difference becomes an anomaly score: a reason to investigate, not proof of fraud. section 8.4.1 section 8.4.2
Turn reconstruction loss into a testable fraud signal:
- Start with scaled transaction features. Filter the training set to legitimate transactions; use labels to select training data, not as targets for the autoencoder. section 8.4.2
- Train the autoencoder to reconstruct those transactions. Check training and validation loss, using early stopping when validation loss stops improving. section 8.4.3
- Reconstruct every transaction in a separate test set. Compute one error per transaction using mean squared error (MSE) or mean absolute error (MAE), then compare the distributions for legitimate and fraudulent cases. section 8.4.4
Calculate a per-transaction score
With `X_test` and its model output `test_reconstructions`, calculate `mse = np.mean((X_test - test_reconstructions) ** 2, axis=1)` or `mae = np.mean(np.abs(X_test - test_reconstructions), axis=1)`. Group these scores by the known test labels for evaluation. In the credit-card example, fraudulent transactions have substantially higher mean and median errors; scatter plots make the separation visible. section 8.4.4 section 8.4.5
Listing 8.12 Calculating reconstruction errors on the full test set
test_reconstructions = autoencoder_model.predict(X_test)
mse_per_instance_test = np.mean(np.power(X_test - test_reconstructions, 2), axis=1) #1
mae_per_instance_test = np.mean(np.abs(X_test - test_reconstructions), axis=1)
reconstruction_error_df_test = pd.DataFrame({
'mse': mse_per_instance_test,
'mae': mae_per_instance_test,
'true_class': y_test #2
})
print("\nReconstruction errors calculated for the test set.")
print("First 5 rows of reconstruction errors with true classes:")
display(reconstruction_error_df_test.head())
print("\nSummary statistics of reconstruction errors, grouped by true class:")
display(reconstruction_error_df_test.groupby('true_class')[['mse', 'mae']].describe()) #3
Inspect the full calculation: model predictions, per-row MSE and MAE, and descriptive statistics grouped by the true test class. section 8.4.4
Live transactions arrive without fraud labels, so the plotted class separation cannot set an operational cutoff by itself. Choose a threshold against trusted normal-error data—for example, a high percentile—and test its alert volume against review capacity and customer-friction limits. Use shadow-mode testing and analyst feedback before relying on alerts. The threshold is a business decision as well as a statistical one. section 8.4.6 section 8.5
