Exact-match accuracy
Exact-match accuracy evaluates outputs by whether they exactly equal their references under specified comparison rules.
Build an AI Agent (From Scratch)
Exact-match evaluation scores a prediction as correct only when it matches the reference answer under specified normalization rules.
- How it is used
- Scores GAIA model answers during per-problem and per-model evaluation.
- How deep it goes
- Defines and implements case and whitespace normalization, but does not describe broader semantic or partial-credit matching.
- Examples
- Uses GAIA final answers, normalized strings, missing predictions, and refusal or error outcomes.
- What you can do after
- Apply it via a Python validator that returns a Boolean correctness result for each prediction-reference pair.
Read how Build an AI Agent (From Scratch) explains exact-match accuracy →
100 Go Mistakes and How to Avoid Them
The equality operator == compares two Go values for equality according to their types and fields.
- How it is used
- Guides equality checks for Go values and diagnoses when using == is invalid or unsafe.
- How deep it goes
- Explains comparable types, compile-time failures, interface panics, and alternatives including deep and explicit comparison.
- Examples
- Uses customer structs with scalar fields, slice fields, interface values, reflect.DeepEqual, and an explicit equal method.
- What you can do after
- Diagnose equality failures and select ==, reflect.DeepEqual, or explicit field comparison appropriately.
Read how 100 Go Mistakes and How to Avoid Them explains exact-match accuracy →
Reinforcement Learning from Human Feedback
Exact match is an evaluation metric that counts an output as correct only when it exactly matches the required answer string or format.
- How it is used
- Evaluates generated answers in post-training and reasoning-era multiple-choice benchmarks.
- How deep it goes
- Defines string or format matching, contrasts log-likelihood scoring, and details sampling, extraction, and formatting failure modes.
- Examples
- Uses MMLU-style answer letters, required answer suffixes, regular-expression extraction, and chain-of-thought responses.
- What you can do after
- Apply it to score generated answers while diagnosing errors caused by sampling, extraction, or rigid formatting.
Read how Reinforcement Learning from Human Feedback explains exact-match accuracy →
Related concepts
- clojure.data/diffused by · clojure.data/diff is a Clojure function that recursively compares two values and returns the parts unique to each value and the parts they share.
- assignmentcontrasts with · Assignment is a programming-language operation that binds or stores an evaluated value in a variable, property, or other assignable location.
- is operatorcontrasts with · The is operator is a Python identity operator that tests whether two operands refer to the same object.
- booleanproduces · A Boolean is a logical type or value representing truth or falsity, conventionally with two states: true and false.
- valueuses · A value is the content, quantity, information, or worth attributed to an entity, property, expression, or outcome.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.