How deeply does Deep Learning with R, Third Edition cover BLEU score?

Defines BLEU and motivates its use but stops short of explaining its n-gram calculation or interpreting score values.

BLEU score

As explained in Deep Learning with R, Third Edition

A BLEU score measures the similarity of machine-translated text to one or more high-quality reference translations using n-gram overlap.

A BLEU score evaluates machine translation by comparing generated text with one or more high-quality reference translations. Unlike next-token accuracy, it accommodates slightly misaligned sequences and better reflects inference, where the decoder generates the target sentence from scratch. section 15.2.2

BLEU is useful because it:

Compares machine-translated text with reference translations. section 15.2.2
Tolerates wording whose sequence is slightly misaligned. section 15.2.2
Addresses the limitations of validation accuracy for autoregressive translation. section 15.2.2

Translation evaluation

A model might generate “te lo voy mañana a decir” for “I will tell you tomorrow,” while a reference uses a different word order. BLEU is designed to assess similarity without requiring exact token-by-token agreement. section 15.2.2

How Deep Learning with R, Third Edition covers it

Cover of Deep Learning with R, Third Edition

liveBook manning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Generative AI in ActionBLEU is an automatic metric that evaluates machine-translated text by measuring its n-gram overlap with a human reference translation.
liveBook manning.com

Sutskever's ListBLEU is an automatic machine-translation metric based on n-gram overlap between generated and reference translations.
liveBook manning.com

LLMs in ProductionBLEU is an evaluation metric for machine translation that measures modified n-gram precision against one or more reference translations.

Compare how 4 books explain BLEU score →

Related concepts

machine translationmeasures · Machine translation is the automated computational conversion of text or speech from one natural language into another.
ROUGEcontrasts with · ROUGE is a family of automatic text-generation evaluation metrics that compare generated text with reference text using lexical overlap, often emphasizing recall.
n-gramuses · An n-gram is a contiguous sequence of n consecutive items, commonly linguistic tokens or characters, drawn from a larger sequence.
mathematical equivalencecontrasts with · Mathematical equivalence is a relation in which different expressions or representations have the same value or meaning under a specified interpretation.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.