liveBook · Manning

BERTScore

BERTScore is an automatic text-generation evaluation metric that compares candidate and reference texts using contextual representations of their tokens.

How 3 Manning books explain it

LLM Evaluation and Alignment, The Foundational Ideas

A text-generation metric that compares semantic representations of candidate and reference text using contextual language-model embeddings.

How it is used
Evaluates generated text against references when valid paraphrases make word overlap too restrictive.
How deep it goes
Explains contextual token embeddings, pairwise cosine similarity, greedy precision and recall, optional IDF weighting, and baseline rescaling.
Examples
Compares “consumers prefer imported cars” with “people like foreign cars” and works through greedy matching for a weather paraphrase.
What you can do after
Evaluate and understand semantic similarity scores while recognizing factual errors, reference dependence, and encoder bias.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains BERTScore →

Building Specialized AI Systems

BERTScore is a text-generation evaluation metric that compares candidate and reference texts using contextual token representations from BERT-like models.

How it is used
Evaluates generated text when semantic equivalence matters more than exact word overlap.
How deep it goes
Explains contextual embeddings and token-level scoring, but defers model-specific implementation details and metric limitations beyond supplementation.
Examples
Uses generated and reference texts, with legal-document and customer-service evaluation needs as domain examples.
What you can do after
Evaluate text-generation quality with semantic similarity scores while supplementing them with expert and application-specific checks.

Read how Building Specialized AI Systems explains BERTScore →

Generative AI in Action

BERTScore is an automatic metric that evaluates generated text by comparing contextual semantic representations of candidate and reference text.

How it is used
Evaluates generated summaries or translations by comparing candidate text with human reference text through semantic similarity.
How deep it goes
Defines contextual embeddings, cosine similarity, token matching, precision, recall, and F1 without mathematical derivation.
Examples
Calculates scores for an automatically generated article summary using the Evaluate library and reports precision, recall, and F1.
What you can do after
Evaluate generated text for semantic similarity and interpret precision, recall, and F1 alongside other evaluation methods.

Read how Generative AI in Action explains BERTScore →

What to know first

Related concepts

Questions about BERTScore

What is BERTScore?

BERTScore is an automatic text-generation evaluation metric that compares candidate and reference texts using contextual representations of their tokens.

Which Manning books explain BERTScore?

LLM Evaluation and Alignment, The Foundational Ideas, Building Specialized AI Systems and Generative AI in Action.

What should I know before learning BERTScore?

Manning books that teach BERTScore usually introduce BLEU score, ROUGE, BERT and evaluation first.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.