Receiver operating characteristic

As explained in Grokking Machine Learning

A receiver operating characteristic curve is a plot of classifier sensitivity against its false-positive rate across decision thresholds.

When a classifier flags someone as sick, how do you balance catching more true cases against wrongly flagging healthy people? A receiver operating characteristic (ROC) curve tracks sensitivity and specificity as the decision threshold changes. section 7.3

Sensitivity is the share of positive cases correctly identified; specificity is the share of negative cases correctly identified. Raising the threshold lowers sensitivity and raises specificity, so each point on the curve represents a different balance. section 7.3

Figure 7.4 Here we can see the ROC curve corresponding to our ongoing example, which gives us a great deal of information on our model. The highlighted dots correspond to the timesteps obtained by moving our threshold from 0 to 1, and each dot is labeled by the timestep. On the horizontal axis we record the sensitivity of the model at each timestep, and on the vertical axis we record the specificity.

See how threshold steps become points on an ROC curve. section 7.3

The area under the curve (AUC) summarizes model performance: values closer to 1 indicate a better model, while values closer to 0.5 indicate a worse one. The curve also helps choose a threshold that suits the relative importance of sensitivity and specificity. section 7.4

Compare threshold choices

For a health test or spam filter, move the threshold through several values and record true positives and true negatives at each step. Compare the resulting sensitivity and specificity to choose a useful balance. section 7.3

How Grokking Machine Learning covers it

How deep it goes

Defines sensitivity and specificity, tracks them as thresholds change, and uses AUC to compare models.

Examples

Uses health diagnosis and spam detection to illustrate classification thresholds, plus a worked sequence of threshold steps.

What it assumes

Assumes positive and negative classifier outcomes; introduces sensitivity and specificity while relating sensitivity to recall.

What you can do after

Evaluate classifier performance and select a threshold that balances sensitivity and specificity for the task.

How it connects

Connects classification thresholds with sensitivity, specificity, false positives, false negatives, and area under the curve.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Statistics Slam DunkA ROC curve plots a binary classifier's true positive rate against its false positive rate across decision thresholds.
• 

  liveBookmanning.com
  Starting Data Analytics with Generative AI and PythonA plot of true positive rate against false positive rate across classification thresholds.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsA receiver operating characteristic curve plots a binary classifier's true positive rate against its false positive rate across decision thresholds.
• 

  liveBookmanning.com
  Grokking StatisticsA receiver operating characteristic curve plots a classifier's true-positive rate against its false-positive rate across decision thresholds.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA receiver operating characteristic curve plots the true positive rate against the false positive rate across classification thresholds.
• 

  liveBookmanning.com
  Financial AI in PracticeA curve that plots a classifier’s true positive rate against its false positive rate across decision thresholds.

Compare how 7 books explain receiver operating characteristic →

Related concepts

• thresholduses · A threshold is a boundary value against which a quantity is compared to determine whether a condition is met or an action or decision is triggered.
• area under the curvemeasured by · The area under a curve is the integral of its values over a specified domain and, in classification evaluation, commonly summarizes performance across decision thresholds.
• recalluses · Recall is the fraction of relevant, positive, or target outcomes that a system successfully identifies, retrieves, or captures.
• specificityuses · Specificity is the proportion of actual negative cases correctly identified as negative, also called the true-negative rate.
• precision-recall curvealternative to · A precision-recall curve is a plot of a classifier’s precision against recall as its decision threshold varies.
• logistic regressionproduced by · Logistic regression is a statistical classification model that estimates categorical outcome probabilities by applying a logistic or softmax function to a linear combination of input features.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.