liveBook · Manning

Recall

As explained in Natural Language Processing in Action, Second Edition

Recall is the proportion of relevant items that a retrieval or classification system successfully identifies.

Recall measures how many relevant items a system successfully finds. For a classifier, it is the number of correctly identified positive cases divided by all actual positive cases: true positives / (true positives + false negatives). In search, a relevant document that goes unmatched is a false negative and lowers recall. section 4.2.1 section 10.3.2

Finding toxic comments

A classifier identified 125 of 326 actually toxic comments, giving it 38.3% recall. Labeling every comment nontoxic would score 87% accuracy on this dataset but have zero recall for toxicity. Labeling every comment toxic would achieve perfect recall while incorrectly flagging many nontoxic comments. The useful balance depends on whether the system alerts moderators or blocks publication. section 4.2.1

>>> from sklearn.metrics import confusion_matrix
>>> confusion_matrix(y_test, lda.predict(X_test))
array([[1261,  913],
       [ 201,  125]], dtype=int64)

Inspect the classifier’s confusion matrix: its toxic-comment row contains 201 missed cases and 125 correctly identified cases, so recall is 125 / (201 + 125). section 4.2.1

Retrieval has the same problem. Lowercasing both a search index and its queries can match documents containing either “Age” or “age,” increasing recall but potentially returning unwanted results. Semantic search addresses another source of missed matches: a query about big cats may otherwise miss documents about lions, tigers, and jaguars that never say “cat.” More matches are not automatically better; check precision alongside recall. section 2.3.2 section 10.3.2

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Defines recall through missed positives and works through classifier counts, without developing a general evaluation procedure.
Examples
Uses toxic-comment predictions, capitalization-sensitive queries, and a big-cat search that misses documents naming individual species.
What it assumes
Assumes familiarity with search or classification tasks while defining recall from concrete counts of identified and missed cases.
What you can do after
Diagnose/debug missed positive cases with a scikit-learn confusion matrix and weigh higher recall against lower precision.
How it connects
Connects recall to false negatives, precision, accuracy, confusion matrices, case normalization, and semantic search.

Also explained in

Compare how 24 books explain recall →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.