A receiver operating characteristic (ROC) curve shows how a binary classifier’s sensitivity changes against its false-positive rate as the decision threshold moves. It helps compare models and choose a threshold when false positives and false negatives have different costs. section 7.0 section 7.12
Build the curve by repeating the same measurements at different thresholds:
- Start with probability predictions, not only positive or negative labels. section 7.14
- Choose thresholds from 0 through 1 and classify predictions above each threshold as positive. section 7.14
- At every threshold, record sensitivity—the proportion of actual positives found—and specificity—the proportion of actual negatives found. section 7.12 section 7.14
- Plot sensitivity against the false-positive rate, which increases as specificity falls. section 7.14
Three threshold choices
Suppose five examples are positive and five are negative. At threshold 0.2, the model finds four positives and three negatives correctly: sensitivity is 4/5 and specificity is 3/5. At threshold 0.5, those values become 3/5 and 4/5. At threshold 0.8, they become 2/5 and 5/5. Lower thresholds catch more positives but create more false positives; higher thresholds do the opposite. section 7.14
Use the curve to match the threshold to the problem. A medical screening model may favor high sensitivity, while a spam detector may favor high specificity. The area under the curve (AUC) summarizes model quality: values nearer 1 are better, while values nearer 0.5 are worse. section 7.16 section 7.18








