ROUGE
ROUGE is a family of automatic text-generation evaluation metrics that compare generated text with reference text using lexical overlap, often emphasizing recall.
Generative AI in Action
ROUGE is a family of automatic metrics that evaluates generated text by measuring overlap with reference text.
- How it is used
- Uses reference-overlap scores to evaluate fine-tuned summarization and translation outputs alongside qualitative and other quantitative measures.
- How deep it goes
- Defines recall, precision, F-measure, ROUGE-N, and ROUGE-L, then demonstrates results without deriving the algorithms mathematically.
- Examples
- Demonstrates an article-summary pipeline with Evaluate and reports ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-Lsum.
- What you can do after
- Evaluate generated summaries or translations by interpreting overlap results and comparing them with semantic and human assessments.
LLMs in Production
ROUGE is a family of automatic evaluation metrics that measure lexical overlap between generated text and reference text, commonly for summarization.
- How it is used
- Evaluates generated summaries against reference summaries when exact matching is too rigid.
- How deep it goes
- Defines N-gram overlap and names ROUGE-L and skip-bigram variants without detailing their algorithms.
- Examples
- Uses Zelda summaries to illustrate ROUGE-1 and ROUGE-L precision, recall, and F-measure.
- What you can do after
- Apply it via the rouge_score Python library to score a target and prediction.
Related concepts
- summarizationmeasures · Summarization is the process of condensing source content into a shorter representation that preserves its important information or meaning.
- BLEU scorecontrasts with · BLEU is an automatic text-generation evaluation metric that compares candidate text with reference text using modified n-gram precision and a brevity penalty.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.