An n-gram is a contiguous sequence of n linguistic units. In text evaluation, overlapping n-grams provide a flexible way to compare generated summaries with reference summaries without requiring exact wording or word order. section 4.2.1
N-grams support several practical comparison strategies:
- ROUGE counts overlapping n-grams between a prediction and a ground-truth summary; greater overlap produces a higher score. section 4.2.1
- The value of n can vary, while related measures compare longest common subsequences or skip-bigrams that preserve word order without requiring adjacency. section 4.2.1
Comparing two Zelda summaries
A reference summary about Link and Hyrule and a differently worded prediction receive nonzero ROUGE scores because they share useful word sequences, even without an exact match. section 4.2.1
Listing 4.1 Using ROUGE
from rouge_score import rouge_scorer
target = "The game 'The Legend of Zelda' follows the adventures of the \
hero Link in the magical world of Hyrule."
prediction = "Link embarks on epic quests and battles evil forces to \
save Princess Zelda and restore peace in the land of Hyrule."
scorer = rouge_scorer.RougeScorer(["rouge1", "rougeL"], use_stemmer=True) #1
scores = scorer.score(target, prediction)
print(scores)
# {'rouge1': Score(precision=0.28571428, recall=0.31578947, fmeasure=0.3),
# 'rougeL': Score(precision=0.238095238, recall=0.26315789, fmeasure=0.25)}
Run the Python rouge-score example to calculate ROUGE-1 and ROUGE-L precision, recall, and F-measure for two summaries. section 4.2.1
