Word overlap measures how many words or word sequences a candidate shares with a high-quality reference. It provides a simple, interpretable proxy for text quality without modeling meaning. section 2.0 section 2.1.1
Evaluation metrics use overlap in several complementary ways:
- BLEU emphasizes precision: it rewards candidate words and phrases that also appear in references. section 2.0
- ROUGE emphasizes recall: it rewards reference content recovered by a candidate, making it useful for summarization. section 2.0
- BLEU combines overlap scores across word-sequence lengths with a geometric mean and length penalty. section 2.0
Translation comparison
For a Chinese news sentence, Candidate 1—“It is a guide to action which ensures that the military always obeys the commands of the party.”—shares phrases such as “It is a guide to action” with the references. Candidate 2 shares fewer recognizable sequences, so overlap helps distinguish the outputs even without understanding Chinese. section 2.1.1
Word overlap does not recognize synonyms or paraphrases: “consumers prefer imported cars” and “people like foreign cars” can express the same meaning while sharing few words. Treat it as a fast sanity check, not a universal quality measure. section 2.0 section 3.5
