Exact-match accuracy
Exact-match accuracy is the proportion of predictions or outputs that equal their reference answers under specified comparison rules.
LLM Customization and Fine-Tuning
Exact match is a metric that assigns a binary score based on whether a generated response exactly matches a reference after normalization.
- How it is used
- Compares a LoRA adapter with its base model on held-out responses.
- How deep it goes
- Defines binary scoring after whitespace normalization and averages per-example results, while leaving normalization implementation details unspecified.
- Examples
- Reports 0% exact match for both base and adapter on long, open-ended IT-support answers.
- What you can do after
- Evaluate and understand exact-match results without mistaking rephrasing for an exact reference match.
Read how LLM Customization and Fine-Tuning explains exact-match accuracy →
Retrieval Augmented Generation, The Foundational Ideas
Exact Match is a question-answering metric that counts an answer as correct only when it exactly matches the reference answer after normalization.
- How it is used
- Evaluates whether question-answering systems produce the exact normalized reference answer.
- How deep it goes
- Defines strict normalized matching and contrasts it with F1, but does not detail the normalization procedure.
- Examples
- Uses the answers “1969” and “Seattle,” plus TriviaQA and Natural Questions benchmark comparisons.
- What you can do after
- Evaluate question-answering systems by checking canonical-answer accuracy and distinguishing exact gains from partial-overlap gains.
Read how Retrieval Augmented Generation, The Foundational Ideas explains exact-match accuracy →
Build AI Drug Discovery Pipelines
Exact match rate is the proportion of reconstructed strings that are identical to their reference strings.
- How it is used
- Evaluates whether a molecular reconstruction reproduces its canonical input string exactly.
- How deep it goes
- Defines the metric, explains canonicalization limits, and contrasts it with validity and structural similarity.
- Examples
- Uses a BasicAutoencoder on the MOSES benchmark, reporting an exact match rate of 0.847.
- What you can do after
- Evaluate/understand reconstruction fidelity while checking whether reference and output strings share canonicalization.
Read how Build AI Drug Discovery Pipelines explains exact-match accuracy →
LLM Evaluation and Alignment, The Foundational Ideas
An exact-match heuristic detects possible data overlap by checking whether identical token or text sequences occur in two data sources.
- How it is used
- Detects lexical overlap for data-contamination audits and diagnoses exact-pattern retrieval behavior.
- How deep it goes
- Defines binary matching and describes n-gram, substring, span, hashing, and indexing methods, but stops short of semantic similarity.
- Examples
- Uses grep-style searches, 13-gram benchmark filtering, duplicate auditing, and verbatim reference-answer overlap.
- What you can do after
- Diagnose lexical retrieval gaps and investigate possible benchmark contamination using exact overlap checks.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains exact-match accuracy →
Rearchitecting LLMs
Exact match is a metric that counts outputs as correct only when they exactly match the reference under the specified comparison rules.
- How it is used
- Measures tool-call response correctness by comparing model outputs with expected responses.
- How deep it goes
- Defines an evaluation harness with normalized, tolerance-aware comparisons, but defers details of the comparison functions.
- Examples
- Compares original and fine-tuned models at 88.9% and 96.3%, and reports scores across pruning and distillation stages.
- What you can do after
- Evaluate tool-call models by scoring extracted responses against expected outputs under configured comparison rules.
Read how Rearchitecting LLMs explains exact-match accuracy →
Build an AI Agent (From Scratch)
Exact-match evaluation scores a prediction as correct only when it matches the reference answer under specified normalization rules.
- How it is used
- Scores GAIA model answers during per-problem and per-model evaluation.
- How deep it goes
- Defines and implements case and whitespace normalization, but does not describe broader semantic or partial-credit matching.
- Examples
- Uses GAIA final answers, normalized strings, missing predictions, and refusal or error outcomes.
- What you can do after
- Apply it via a Python validator that returns a Boolean correctness result for each prediction-reference pair.
Read how Build an AI Agent (From Scratch) explains exact-match accuracy →
100 Go Mistakes and How to Avoid Them
The equality operator == compares two Go values for equality according to their types and fields.
- How it is used
- Guides equality checks for Go values and diagnoses when using == is invalid or unsafe.
- How deep it goes
- Explains comparable types, compile-time failures, interface panics, and alternatives including deep and explicit comparison.
- Examples
- Uses customer structs with scalar fields, slice fields, interface values, reflect.DeepEqual, and an explicit equal method.
- What you can do after
- Diagnose equality failures and select ==, reflect.DeepEqual, or explicit field comparison appropriately.
Read how 100 Go Mistakes and How to Avoid Them explains exact-match accuracy →
Reinforcement Learning from Human Feedback
Exact match is an evaluation metric that counts an output as correct only when it exactly matches the required answer string or format.
- How it is used
- Evaluates generated answers in post-training and reasoning-era multiple-choice benchmarks.
- How deep it goes
- Defines string or format matching, contrasts log-likelihood scoring, and details sampling, extraction, and formatting failure modes.
- Examples
- Uses MMLU-style answer letters, required answer suffixes, regular-expression extraction, and chain-of-thought responses.
- What you can do after
- Apply it to score generated answers while diagnosing errors caused by sampling, extraction, or rigid formatting.
Read how Reinforcement Learning from Human Feedback explains exact-match accuracy →
Also covered in
- Clojure: The Essential Reference
- MongoDB 8.0 in Action, Third Edition
- The Well-Grounded Rubyist, Fourth Edition
- Get Programming: Learn to code with Python
- Python How-To
- Learn SQL in a Month of Lunches
- Learn AI-Assisted Python Programming, Second Edition
- Just Use Postgres!
- Kubernetes in Action, Second Edition
- 100 C++ Mistakes and How to Avoid Them
- Natural Language Processing in Action, Second Edition
- 100 Java Mistakes and How to Avoid Them
What to know first
- MapA map is an associative data structure that stores key-value associations and supports retrieving values by key.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- listA list is an ordered collection of elements arranged in a sequence.
- SQLSQL is a standardized declarative language for defining, querying, and manipulating data in relational database systems.
Related concepts
- clojure.data/diffused by · clojure.data/diff is a Clojure function that recursively compares two values and returns the parts unique to each value and the parts they share.
- token-F1contrasts with · An overlap-based evaluation metric for text generation that measures partial word overlap between generated and reference responses.
- assignmentcontrasts with · Assignment is a programming-language operation that binds or stores an evaluated value in a variable, property, or other assignable location.
- booleanproduces · A Boolean is a logical value or data type representing truth or falsity, conventionally with the two values true and false.
- is operatorcontrasts with · The is operator is a Python identity operator that tests whether two operands refer to the same object.
- arbitrary-precision integerkind of · An integer representation that can accommodate values beyond fixed-width limits by expanding storage as needed, without loss of integer precision.
Questions about exact-match accuracy
What is exact-match accuracy?
Exact-match accuracy is the proportion of predictions or outputs that equal their reference answers under specified comparison rules.
Which Manning books explain exact-match accuracy?
LLM Customization and Fine-Tuning, Retrieval Augmented Generation, The Foundational Ideas, Build AI Drug Discovery Pipelines, LLM Evaluation and Alignment, The Foundational Ideas, Rearchitecting LLMs, Build an AI Agent (From Scratch), 100 Go Mistakes and How to Avoid Them and Reinforcement Learning from Human Feedback.
What should I know before learning exact-match accuracy?
Manning books that teach exact-match accuracy usually introduce Map, language model, list and SQL first.
What is the difference between exact-match accuracy and token-F1?
Exact-match accuracy is the proportion of predictions or outputs that equal their reference answers under specified comparison rules. An overlap-based evaluation metric for text generation that measures partial word overlap between generated and reference responses.
What is exact-match accuracy a kind of?
Exact-match accuracy is a kind of arbitrary-precision integer. An integer representation that can accommodate values beyond fixed-width limits by expanding storage as needed, without loss of integer precision.
What is exact-match accuracy also called?
Exact-match accuracy is also called EM.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.







