Suppose a medical test labels sick patients as positive. How many of the sick patients does it actually identify? Recall answers that question: it is the proportion of positive-labeled cases correctly predicted as positive. It is especially useful when missing a positive case—a false negative—is costly. section 7.2 section 7.4
Count the true positives (TP), then divide by all cases that really are positive: Recall = TP / (TP + FN). Here FN means false negatives: positive cases predicted as negative. You can find both counts in the positive-labeled row of a confusion matrix. section 13.4 section 7.2
A high-accuracy test that misses every sick patient
One test predicts that all 1,000 patients are healthy. It correctly labels 990 healthy patients, but misses all 10 sick patients. Its recall is 0 / (0 + 10) = 0, despite its 99% accuracy. A second test identifies 8 of the 10 sick patients, giving it recall of 8 / (8 + 2) = 0.8; it also incorrectly flags 48 healthy patients. The better choice depends on which errors matter most. section 7.2 section 7.4
Use recall alongside other measures, not by itself:
- Precision asks a different question: of cases predicted positive, how many really are positive? F1-score combines precision and recall. section 7.4
- Sensitivity, also called the true positive rate, is the same measure as recall. Raising a classification threshold can lower sensitivity while raising specificity; an ROC curve helps examine that trade-off. section 7.3
Figure 7.3 The effects of moving the threshold on the sensitivity and the specificity. On the left, we have a model with a low threshold; in the middle, we have one with a medium threshold; and on the right, we have one with a high threshold. For each of the models, there are five positive and five negative points. Each model is represented by the vertical line. The model predicts that the points to the right of the line are positive and those to the left are negative. For each of the models, we’ve counted the number of true positives and true negatives, that is, the number of positive and negative points that have been correctly predicted. We have used those to calculate the sensitivity and the specificity. Note that as we increase the threshold (i.e., as we move the vertical line from left to right), the sensitivity goes down and the specificity goes up.
Compare the three threshold positions: as the line moves right, fewer positive points are identified, so sensitivity—recall—falls. section 7.3
Try it: a confusion matrix shows 120 sick patients predicted sick and 22 predicted healthy. Calculate recall, then ask whether those 22 missed cases make the model suitable for its intended use. section 7.5
