liveBook · Manning

Recall

As explained in LLMs in Production

Recall is the proportion of reference items that are correctly identified by a prediction.

Exact string equality can penalize a valid summary that uses different wording. Recall measures the proportion of reference items correctly identified; in ROUGE, overlap replaces exact matching. section 4.2.1

ROUGE applies recall-oriented comparison to generated and ground-truth summaries using n-grams, longest common subsequences, or skip-bigrams. Greater overlap produces a higher score. section 4.2.1

Comparing two Zelda summaries

A ROUGE-1 comparison reports recall 0.3158, precision 0.2857, and F-measure 0.3000; ROUGE-L reports recall 0.2632. section 4.2.1

Listing 4.1 Using ROUGE
from rouge_score import rouge_scorer

target = "The game 'The Legend of Zelda' follows the adventures of the \
    hero Link in the magical world of Hyrule."
prediction = "Link embarks on epic quests and battles evil forces to \
    save Princess Zelda and restore peace in the land of Hyrule."

scorer = rouge_scorer.RougeScorer(["rouge1", "rougeL"], use_stemmer=True)   #1
scores = scorer.score(target, prediction)
print(scores)
# {'rouge1': Score(precision=0.28571428, recall=0.31578947, fmeasure=0.3),
# 'rougeL': Score(precision=0.238095238, recall=0.26315789, fmeasure=0.25)}

Run the supplied Python listing to calculate ROUGE-1 and ROUGE-L scores, including recall for generated text against a target summary. section 4.2.1

How LLMs in Production covers it

How deep it goes
Defines recall as reference-item coverage and applies it to ROUGE overlap, without presenting a mathematical derivation.
Examples
Demonstrates ROUGE-1 and ROUGE-L on two Zelda summaries, reporting recall alongside precision and F-measure.
What it assumes
Assumes ground-truth and generated text, then builds from exact matching toward overlap-based comparison.
What you can do after
Apply it via rouge_score to inspect how much reference-summary content a generated summary overlaps.
How it connects
Connects recall to ROUGE, n-grams, longest common subsequences, skip-bigrams, precision, and F-measure.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

Compare how 24 books explain recall →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.