liveBook · Manning

Recall

As explained in Machine Learning System Design

The proportion of actual positive cases that a model correctly predicts as positive.

Suppose a fraud detector catches 80 of 100 fraudulent transactions. Its recall is 80%: of the cases that really were positive, it identified 80. Recall is also called the true positive rate (TPR). Calculate it as true positives divided by true positives plus false negatives. The missed frauds matter because recall counts what the model failed to catch, not just what it flagged. section 5.2

Same recall, very different workload

Two detectors each catch 80 of 100 fraudulent transactions among 100,000 genuine ones, so both have 80% recall. One also flags 20 genuine transactions; the other flags 920. Recall alone cannot distinguish their false-alarm burden. Check precision and false positive rate alongside it, especially when positive cases are rare. Changing the number of fraud cases while keeping each detector's recall at 80% also changes precision. section 5.2

Choose recall in light of the decision it supports. A high value says that few actual positives were missed; it does not establish that positive predictions are trustworthy or that the system will meet its business goal. The metric used to assess a model may differ from its training loss, and choosing an unsuitable metric can steer development toward poor real-world results. section 5.2

Recall within a search shortlist

In a photo-search experiment, a retrospective check found that 97% of all clicked photos appeared in the new model's top 1,000 results. This Recall@1000 finding suggested a possible next step: a two-stage pipeline with a retrieval stage. It was an observation about the shortlist, not a substitute for the experiment's user and business metrics. section 12.4.2

Do not confuse classifier recall with the sensitivity of a statistical test. In A/B-test simulations, sensitivity means the estimated probability that a test detects a real difference; it is assessed by repeatedly simulating an uplift and checking how often the null hypothesis is rejected. section 12.2.5

How Machine Learning System Design covers it

How deep it goes
Defines recall through true positives and false negatives, then compares simulated results; stops short of deriving a training objective.
Examples
Uses fraud-detector simulations with unequal false alarms and a photo-search check of clicked results in the top 1,000.
What it assumes
Assumes familiarity with predicted and actual class labels while showing how to count true positives and false negatives.
What you can do after
Evaluate/understand it by comparing missed positives with false alarms and interpreting Recall@1000 as evidence for a possible retrieval stage.
How it connects
Connects recall to precision, false positive rate, class imbalance, training loss and retrieval pipelines.

Also explained in

Compare how 24 books explain recall →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.