liveBook · Manning

ROC curve

As explained in Grokking Machine Learning, Second Edition

A receiver operating characteristic curve is a plot of a binary classifier's true-positive rate against its false-positive rate as the decision threshold varies.

A receiver operating characteristic (ROC) curve shows how a binary classifier’s sensitivity changes against its false-positive rate as the decision threshold moves. It helps compare models and choose a threshold when false positives and false negatives have different costs. section 7.0 section 7.12

Build the curve by repeating the same measurements at different thresholds:

  • Start with probability predictions, not only positive or negative labels. section 7.14
  • Choose thresholds from 0 through 1 and classify predictions above each threshold as positive. section 7.14
  • At every threshold, record sensitivity—the proportion of actual positives found—and specificity—the proportion of actual negatives found. section 7.12 section 7.14
  • Plot sensitivity against the false-positive rate, which increases as specificity falls. section 7.14

Three threshold choices

Suppose five examples are positive and five are negative. At threshold 0.2, the model finds four positives and three negatives correctly: sensitivity is 4/5 and specificity is 3/5. At threshold 0.5, those values become 3/5 and 4/5. At threshold 0.8, they become 2/5 and 5/5. Lower thresholds catch more positives but create more false positives; higher thresholds do the opposite. section 7.14

Use the curve to match the threshold to the problem. A medical screening model may favor high sensitivity, while a spam detector may favor high specificity. The area under the curve (AUC) summarizes model quality: values nearer 1 are better, while values nearer 0.5 are worse. section 7.16 section 7.18

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Explains threshold sweeps, sensitivity, specificity, plotted trade-offs, AUC, and concrete calculations, without implementation code.
Examples
Uses five positive and five negative points evaluated at thresholds 0.2, 0.5, and 0.8.
What it assumes
Assumes binary classification and introduces sensitivity and specificity before constructing the curve.
What you can do after
Evaluate models and choose a threshold that fits the required sensitivity-specificity balance.
How it connects
Connects confusion matrix errors, sensitivity, specificity, precision, recall, classification thresholds, and AUC.

Also explained in

Compare how 9 books explain ROC curve →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.