liveBook · Manning

Greedy matching

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Greedy matching is a matching strategy that selects the best available correspondence for each item without globally optimizing all correspondences jointly.

Greedy matching selects the best semantic correspondence independently for each token. In BERTScore, every candidate token searches all reference tokens for its highest similarity, so sentences can differ in length, order, and wording. section 3.2.4

A BERTScore comparison follows these steps:

  • Embed both sentences with a pretrained encoder such as RoBERTa, producing contextual vectors for every token. section 3.2.4
  • Calculate cosine similarity for every candidate–reference token pair. section 3.2.4
  • For recall, find each reference token’s best candidate match; for precision, find each candidate token’s best reference match. section 3.2.4
  • Average the selected similarities, optionally weighting distinctive words more heavily with inverse document frequency. section 3.2.4

Paraphrase matching

For the reference “The weather is cold today” and candidate “It is freezing today,” “cold” can match “freezing” through contextual similarity. The example’s unweighted BERTScore recall is 0.759. section 3.2.4

Figure 3.7 Computing BERTScore. Given the reference and candidate, we compute embeddings and pairwise cosine similarity. The heatmap highlights the greedy matching, where each reference token is matched to its best candidate token. Optional importance weighting emphasizes less common words over common ones. And we will scale the score to be more evenly distributed between 0 and 1.

The diagram traces embeddings, pairwise cosine similarities, greedy token matches, optional importance weighting, and score scaling. section 3.2.4

Greedy matching improves semantic coverage for paraphrases and reordered text, but it remains reference-based. A high similarity score does not establish factual accuracy or overall translation quality. section 3.2.5

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines embeddings, pairwise cosine similarities, recall, precision, and IDF weighting; it stops short of judging factual correctness.
Examples
Uses “The weather is cold today” versus “It is freezing today,” including a 0.759 unweighted recall example.
What it assumes
Assumes familiarity with text embeddings and cosine similarity while introducing token-level semantic matching.
What you can do after
Apply it via BERTScore to compare paraphrased or reordered candidate text against a reference.
How it connects
Connects contextual embeddings, cosine similarity, BERTScore, IDF weighting, baseline rescaling, and reference-based evaluation.

Also explained in

Compare how 4 books explain greedy matching →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.