liveBook · Manning

Exact-match accuracy

Also called EM

Exact-match accuracy is the proportion of predictions or outputs that equal their reference answers under specified comparison rules.

How 8 Manning books explain it

LLM Customization and Fine-Tuning

Exact match is a metric that assigns a binary score based on whether a generated response exactly matches a reference after normalization.

How it is used
Compares a LoRA adapter with its base model on held-out responses.
How deep it goes
Defines binary scoring after whitespace normalization and averages per-example results, while leaving normalization implementation details unspecified.
Examples
Reports 0% exact match for both base and adapter on long, open-ended IT-support answers.
What you can do after
Evaluate and understand exact-match results without mistaking rephrasing for an exact reference match.

Read how LLM Customization and Fine-Tuning explains exact-match accuracy →

Retrieval Augmented Generation, The Foundational Ideas

Exact Match is a question-answering metric that counts an answer as correct only when it exactly matches the reference answer after normalization.

How it is used
Evaluates whether question-answering systems produce the exact normalized reference answer.
How deep it goes
Defines strict normalized matching and contrasts it with F1, but does not detail the normalization procedure.
Examples
Uses the answers “1969” and “Seattle,” plus TriviaQA and Natural Questions benchmark comparisons.
What you can do after
Evaluate question-answering systems by checking canonical-answer accuracy and distinguishing exact gains from partial-overlap gains.

Read how Retrieval Augmented Generation, The Foundational Ideas explains exact-match accuracy →

Build AI Drug Discovery Pipelines

Exact match rate is the proportion of reconstructed strings that are identical to their reference strings.

How it is used
Evaluates whether a molecular reconstruction reproduces its canonical input string exactly.
How deep it goes
Defines the metric, explains canonicalization limits, and contrasts it with validity and structural similarity.
Examples
Uses a BasicAutoencoder on the MOSES benchmark, reporting an exact match rate of 0.847.
What you can do after
Evaluate/understand reconstruction fidelity while checking whether reference and output strings share canonicalization.

Read how Build AI Drug Discovery Pipelines explains exact-match accuracy →

LLM Evaluation and Alignment, The Foundational Ideas

An exact-match heuristic detects possible data overlap by checking whether identical token or text sequences occur in two data sources.

How it is used
Detects lexical overlap for data-contamination audits and diagnoses exact-pattern retrieval behavior.
How deep it goes
Defines binary matching and describes n-gram, substring, span, hashing, and indexing methods, but stops short of semantic similarity.
Examples
Uses grep-style searches, 13-gram benchmark filtering, duplicate auditing, and verbatim reference-answer overlap.
What you can do after
Diagnose lexical retrieval gaps and investigate possible benchmark contamination using exact overlap checks.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains exact-match accuracy →

Rearchitecting LLMs

Exact match is a metric that counts outputs as correct only when they exactly match the reference under the specified comparison rules.

How it is used
Measures tool-call response correctness by comparing model outputs with expected responses.
How deep it goes
Defines an evaluation harness with normalized, tolerance-aware comparisons, but defers details of the comparison functions.
Examples
Compares original and fine-tuned models at 88.9% and 96.3%, and reports scores across pruning and distillation stages.
What you can do after
Evaluate tool-call models by scoring extracted responses against expected outputs under configured comparison rules.

Read how Rearchitecting LLMs explains exact-match accuracy →

Build an AI Agent (From Scratch)

Exact-match evaluation scores a prediction as correct only when it matches the reference answer under specified normalization rules.

How it is used
Scores GAIA model answers during per-problem and per-model evaluation.
How deep it goes
Defines and implements case and whitespace normalization, but does not describe broader semantic or partial-credit matching.
Examples
Uses GAIA final answers, normalized strings, missing predictions, and refusal or error outcomes.
What you can do after
Apply it via a Python validator that returns a Boolean correctness result for each prediction-reference pair.

Read how Build an AI Agent (From Scratch) explains exact-match accuracy →

100 Go Mistakes and How to Avoid Them

The equality operator == compares two Go values for equality according to their types and fields.

How it is used
Guides equality checks for Go values and diagnoses when using == is invalid or unsafe.
How deep it goes
Explains comparable types, compile-time failures, interface panics, and alternatives including deep and explicit comparison.
Examples
Uses customer structs with scalar fields, slice fields, interface values, reflect.DeepEqual, and an explicit equal method.
What you can do after
Diagnose equality failures and select ==, reflect.DeepEqual, or explicit field comparison appropriately.

Read how 100 Go Mistakes and How to Avoid Them explains exact-match accuracy →

Reinforcement Learning from Human Feedback

Exact match is an evaluation metric that counts an output as correct only when it exactly matches the required answer string or format.

How it is used
Evaluates generated answers in post-training and reasoning-era multiple-choice benchmarks.
How deep it goes
Defines string or format matching, contrasts log-likelihood scoring, and details sampling, extraction, and formatting failure modes.
Examples
Uses MMLU-style answer letters, required answer suffixes, regular-expression extraction, and chain-of-thought responses.
What you can do after
Apply it to score generated answers while diagnosing errors caused by sampling, extraction, or rigid formatting.

Read how Reinforcement Learning from Human Feedback explains exact-match accuracy →

Also covered in

What to know first

Related concepts

Questions about exact-match accuracy

What is exact-match accuracy?

Exact-match accuracy is the proportion of predictions or outputs that equal their reference answers under specified comparison rules.

Which Manning books explain exact-match accuracy?

LLM Customization and Fine-Tuning, Retrieval Augmented Generation, The Foundational Ideas, Build AI Drug Discovery Pipelines, LLM Evaluation and Alignment, The Foundational Ideas, Rearchitecting LLMs, Build an AI Agent (From Scratch), 100 Go Mistakes and How to Avoid Them and Reinforcement Learning from Human Feedback.

What should I know before learning exact-match accuracy?

Manning books that teach exact-match accuracy usually introduce Map, language model, list and SQL first.

What is the difference between exact-match accuracy and token-F1?

Exact-match accuracy is the proportion of predictions or outputs that equal their reference answers under specified comparison rules. An overlap-based evaluation metric for text generation that measures partial word overlap between generated and reference responses.

What is exact-match accuracy a kind of?

Exact-match accuracy is a kind of arbitrary-precision integer. An integer representation that can accommodate values beyond fixed-width limits by expanding storage as needed, without loss of integer precision.

What is exact-match accuracy also called?

Exact-match accuracy is also called EM.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.