liveBook · Manning

ROUGE

Also called Recall-Oriented Understudy for Gisting Evaluation

ROUGE is a family of automatic text-generation evaluation metrics that compare generated text with reference text using lexical overlap, often emphasizing recall.

How 3 Manning books explain it

LLM Evaluation and Alignment, The Foundational Ideas

ROUGE is a family of automatic text-generation metrics that compares system output with reference text using overlapping units such as n-grams or subsequences.

How it is used
Measures reference overlap for summarization and provides a low-cost signal for regression monitoring.
How deep it goes
Explains clipped n-gram recall, subsequence variants, and precision-recall balance with a worked calculation.
Examples
Calculates unigram and bigram recall for a short summary and describes nightly ROUGE-L regression testing.
What you can do after
Evaluate and understand reference overlap, choose complementary variants, and interpret score changes as prompts for further review.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains ROUGE →

Generative AI in Action

ROUGE is a family of automatic metrics that evaluates generated text by measuring overlap with reference text.

How it is used
Uses reference-overlap scores to evaluate fine-tuned summarization and translation outputs alongside qualitative and other quantitative measures.
How deep it goes
Defines recall, precision, F-measure, ROUGE-N, and ROUGE-L, then demonstrates results without deriving the algorithms mathematically.
Examples
Demonstrates an article-summary pipeline with Evaluate and reports ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-Lsum.
What you can do after
Evaluate generated summaries or translations by interpreting overlap results and comparing them with semantic and human assessments.

Read how Generative AI in Action explains ROUGE →

LLMs in Production

ROUGE is a family of automatic evaluation metrics that measure lexical overlap between generated text and reference text, commonly for summarization.

How it is used
Evaluates generated summaries against reference summaries when exact matching is too rigid.
How deep it goes
Defines N-gram overlap and names ROUGE-L and skip-bigram variants without detailing their algorithms.
Examples
Uses Zelda summaries to illustrate ROUGE-1 and ROUGE-L precision, recall, and F-measure.
What you can do after
Apply it via the rouge_score Python library to score a target and prediction.

Read how LLMs in Production explains ROUGE →

Also covered in

What to know first

Related concepts

Questions about ROUGE

What is ROUGE?

ROUGE is a family of automatic text-generation evaluation metrics that compare generated text with reference text using lexical overlap, often emphasizing recall.

Which Manning books explain ROUGE?

LLM Evaluation and Alignment, The Foundational Ideas, Generative AI in Action and LLMs in Production.

What should I know before learning ROUGE?

Manning books that teach ROUGE usually introduce machine translation, summarization and human-in-the-loop first.

What is the difference between ROUGE and BLEU score?

ROUGE is a family of automatic text-generation evaluation metrics that compare generated text with reference text using lexical overlap, often emphasizing recall. BLEU is an automatic text-generation evaluation metric that compares candidate text with reference text using modified n-gram precision and a brevity penalty.

What kinds of ROUGE are there?

Kinds of ROUGE include ROUGE-L, ROUGE-1 and ROUGE-2.

What is ROUGE also called?

ROUGE is also called Recall-Oriented Understudy for Gisting Evaluation.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.