liveBook · Manning

Precision-recall curve

As explained in Fight Fraud with Machine Learning

A precision-recall curve is a plot showing the trade-off between precision and recall across a range of classification thresholds.

A precision-recall curve shows how a fraud classifier’s precision and recall change as its probability threshold moves. Convert predicted probabilities into binary fraud or genuine labels across thresholds from 0 to 1, calculate both metrics at each point, and plot precision against recall. section 1.6.4

Use the curve to make two practical evaluation decisions:

  • Select an operating threshold by choosing the precision-recall combination that fits the application’s tolerance for missed fraud and false alarms. section 1.6.4
  • Compare models without committing to one threshold by comparing the area under their precision-recall curves; the larger area indicates the stronger system. section 1.6.4 section 1.7

Setting a fraud threshold

For a fraud-detection model, inspect the PR curve and select the threshold that achieves maximum recall while maintaining at least 99.99% precision. Save that threshold with the trained model when deploying predictions. section 3.9

Figure 1.22 Precision-recall curve with hypothetical threshold values pointing to two different sets of precision and recall values.

Use the threshold-labeled curve to connect alternative operating points with their corresponding precision and recall values. section 1.6.4

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains threshold sweeping, plotting, threshold-free area comparison, and an operational precision constraint.
Examples
Uses hypothetical threshold points, competing fraud systems, and a 99.99% minimum-precision selection rule.
What it assumes
Assumes binary fraud labels and probability outputs, while treating class imbalance as a reason to prefer precision and recall over accuracy.
What you can do after
Evaluate fraud models without a fixed threshold and select an operating threshold from the precision-recall trade-off.
How it connects
Connects precision, recall, classification thresholds, area under the curve, class imbalance, and fraud-detection systems.

Also explained in

Compare how 5 books explain precision-recall curve →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.