liveBook · Manning

Area under the ROC curve

As explained in Grokking Machine Learning, Second Edition

AUC is the area under a receiver operating characteristic curve, measuring how well a binary classifier distinguishes between its classes across decision thresholds.

AUC measures how well a binary classifier separates its two classes across decision thresholds. It is the area beneath the receiver operating characteristic (ROC) curve: values closer to 1 indicate stronger separation, while 0.5 means performance no better than random guessing. section 7.15 section 7.18

Comparing three models

Imagine three ROC curves drawn on a grid whose squares each measure 0.2 by 0.2. Counting the squares beneath them gives AUC values of 0.52, 0.72, and 1. The model scoring 0.72 separates the classes better than the model scoring 0.52; the model scoring 1 is perfect. An AUC below 0.5 can be improved by flipping the positive and negative predictions. section 7.15

Use AUC alongside the ROC curve to understand threshold trade-offs:

  • Increasing the decision threshold decreases sensitivity and increases specificity. section 7.18
  • The ROC curve records sensitivity and specificity at each threshold, helping select a balance suited to the model's needs. section 7.18
Figure 7.5 In this figure, we can see that AUC, or area under the curve, is a good metric to determine how good a model is. The higher the AUC, the better the model. On the left, we have a bad model with an AUC of 0.52. In the middle, we have a good model with an AUC of 0.72. On the right, we have a great model with an AUC of 1.

Compare ROC curves visually: the figure contrasts AUC values of 0.52, 0.72, and 1. section 7.15

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Calculates ROC area through a grid example and interprets benchmark values, without developing a formal computation procedure.
Examples
Compares three ROC curves with AUC values of 0.52, 0.72, and 1 using counted grid squares.
What it assumes
Uses sensitivity, specificity, ROC curves, decision thresholds, and binary classification without establishing their foundations here.
What you can do after
Evaluate a binary classifier and choose thresholds by reading ROC behavior and comparing AUC values.
How it connects
Connects AUC with ROC curves, sensitivity, specificity, decision thresholds, and binary classification.

Also explained in

Compare how 14 books explain area under the ROC curve →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.