liveBook · Manning

Accuracy

As explained in Fight Fraud with Machine Learning

Accuracy is the proportion of predictions that match the actual class labels.

Accuracy is the proportion of predictions that match the actual class labels. For 85 correct predictions out of 100, accuracy is 85%. section 1.6.1

Interpret accuracy alongside class distribution and other evaluation measures:

  • Use accuracy cautiously when classes are imbalanced: predicting every email as genuine can produce 99% accuracy when genuine-to-fraud examples occur at a 99:1 ratio. section 1.6.1
  • Use accuracy more confidently when binary classes are approximately balanced, while also examining precision, recall, the confusion matrix, and the PR curve for fraud detection. section 1.6.1 section 1.7
  • Compute it by comparing binarized predictions with ground-truth labels and dividing correct predictions by the evaluated sample count. section 8.2.3 section 9.2.3

Model-testing example

A voice-faking classifier evaluates unseen FoR audio clips and reports about 70% accuracy, better than the 50% random baseline. An MLP detecting illicit versus licit Bitcoin transactions reports 93.28% accuracy alongside a 93.77% F1 score. section 8.2.3 section 9.2.3

model.eval()
correct = 0
total = 0
with torch.no_grad():
    for inputs, labels in test_loader:
        inputs, labels = inputs.to(device), labels.to(device)
        outputs = model(inputs)
        _, predicted = torch.max(outputs, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()
print(f'Test Accuracy: {100 * correct / total} %')

PyTorch test-loop code shows evaluation mode, disabled gradients, class prediction, correct-count accumulation, and test-accuracy calculation. section 8.2.3

How Fight Fraud with Machine Learning covers it

How deep it goes
Defines and calculates the proportion of correct predictions, implements test-loop evaluation, and stops short of treating it as sufficient for imbalanced fraud data.
Examples
Uses fraud emails, fake-voice clips, and illicit Bitcoin transactions to demonstrate accuracy calculations and practical limitations.
What it assumes
Assumes labeled binary predictions and ground-truth classes while teaching the metric calculation and its class-imbalance limitation.
What you can do after
Evaluate classification models via PyTorch and judge when accuracy requires support from imbalance-aware metrics.
How it connects
Connects accuracy with binary classification, class imbalance, precision, recall, F1 score, confusion matrices, and PR curves.

Also explained in

Compare how 49 books explain accuracy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.