Chapter 3: Evaluating reasoning models

Introduction

The chapter introduction defines evaluation as quantitative testing against reference answers and identifies math-answer verification as the chapter’s approach.

• Quantitative Model Evaluation: Explain why convincing-sounding responses are not enough to establish that a model solves problems correctly, and describe evaluation here as scoring outputs against reference answers across many examples.
  Why it matters: This distinction makes correctness, rather than plausible wording, the basis for judging the model.
• Math Answer Verification: Describe the chapter’s verification-based approach at a high level: compare a model’s math answer with a reference solution using a calculator-like implementation to check correctness.
  Why it matters: It gives the chapter a concrete way to judge math answers rather than relying on how they read.

Key terms: evaluation, reference answers, verification-based method, reference solutions, calculator-like implementation

3.1 Building a math verifier

This section positions math verifiers among LLM evaluation methods, explains why checking math answers is useful, and identifies the approach’s limits and later role in training.

• LLM evaluation approaches: Distinguish the four common evaluation approaches named here and the broader contrast between benchmark-based and judgment-based evaluation.
  Why it matters: This distinction explains where the chapter’s verifier-based evaluation fits.
• Math verifier answer checking: Explain how a math verifier assesses an LLM answer and why math problems make this approach useful for evaluating reasoning models.
  Why it matters: It connects a model’s potentially lengthy reasoning response to a checkable measure of correctness.
• Verifier applicability and tradeoffs: Explain where verifier-based evaluation works and what costs or limitations it introduces.
  Why it matters: It prevents treating a useful math-evaluation method as a universal solution.
• Verifier feedback in training: State the additional role verifiers can play beyond measuring model performance.
  Why it matters: It shows why answer checking remains useful after an evaluation run.

Key terms: multiple choice, verifiers, leaderboards, LLM judges, benchmark-based evaluation, judgment-based evaluation, verifier, extracted answer, reference solution, step-by-step reasoning

3.2 Loading a pretrained model to generate text

The section loads a pretrained model and generates a math answer in a form that can be inspected now and processed by the verifier later.

• Base and reasoning model choices: Distinguish the default pretrained base model from the optional official Qwen3 reasoning model, and explain why the latter is offered as a reference point.
  Why it matters: Keeping the model-loading choice clear makes later evaluation comparisons meaningful.
• Streaming and storing generated tokens: Trace how the math prompt and generated tokens become a stored answer string, and explain why the tokens are both printed and collected.
  Why it matters: The verifier needs a complete answer string, not just output displayed while the model runs.
• Base model reasoning-like output: Explain what the example output shows about the base model without treating it as a model trained with the later reasoning methods.
  Why it matters: A step-by-step-looking answer from the base model should not be confused with the improvements the book will evaluate later.

Key terms: load_model_and_tokenizer, base model, official Qwen3 reasoning model, WHICH_MODEL, token IDs, streaming, all_tokens, decode, reasoning-model-like explanation, chain-of-thought data

3.3 Implementing a wrapper for easier text generation

This section introduces a reusable wrapper that turns a prompt into a generated string without repeating input-preparation steps.

• Reusable text generation wrapper: Describe why the wrapper is useful and trace how it turns a prompt into a final string. Explain what the optional verbose flag shows.
  Why it matters: The wrapper makes repeated model generation calls simpler while still returning the generated text.

Key terms: wrapper, tokenizes, streams, decodes, verbose flag

3.4 Extracting the final answer box

This section builds utilities that extract a final answer from model output, preferring the last boxed expression and using configurable fallbacks when no box is found.

• Boxed Answer Convention: Explain why boxed answers are useful for automatically evaluating math responses and why the earlier model response might have used that format without being instructed to.
  Why it matters: A predictable answer format makes large-scale scoring easier.
• Last Boxed Content Extraction: Describe how get_last_boxed selects and extracts an answer from model output, including how it handles braces inside the box.
  Why it matters: The answer must be isolated from the rest of the response before it can be checked.
• Missing Box Fallback Choices: Distinguish the three extract_final_candidate fallback settings and describe how a numeric candidate is found when no boxed answer is available.
  Why it matters: Fallback choices determine what can be scored when a response does not contain a usable box.
• Deterministic Extraction Rationale: Justify using a small parser and regex utility rather than another LLM to extract the answer, and distinguish extraction from later correctness checking.
  Why it matters: Reliable extraction keeps a separate model’s variability out of the evaluation pipeline.

Key terms: answer box, boxed answers, MATH-500, get_last_boxed, \boxed, nested braces, extract_final_candidate, number_then_full, number_only, none

3.5 Normalizing the extracted answer

This section explains why extracted math answers need normalization and how `normalize_text` converts varied formatting into a consistent string for later comparison.

• Purpose of answer normalization: Explain why an extracted answer is normalized before it is checked against a reference answer: the same value can appear in different formats, so formatting must not prevent a consistent comparison.
  Why it matters: A formatting difference should not, by itself, keep a model answer from being checked against the reference.
• Normalize text transformations: Describe how `normalize_text` rewrites an extracted answer into a standardized, LaTeX-free string, including removal of formatting clutter and conversion of common math structures.
  Why it matters: These rewrites give later checking code a consistent string rather than many LaTeX variations of an answer.

Key terms: normalization, canonicalization, extracted answer, normalize_text, LaTeX, standardized format, calculator-style form

3.6 Verifying mathematical equivalence

This section builds a SymPy-based check for whether an LLM’s extracted answer is mathematically equivalent to a ground-truth answer, then identifies its limitation with tuples.

• Mathematical versus string equality: Explain why comparing answer strings with Python’s == operator is insufficient for verifying mathematical answers.
  Why it matters: A verifier should judge the answer’s mathematical value, not just how it was written.
• SymPy answer parsing: Describe how sympy_parser prepares an answer expression for mathematical comparison.
  Why it matters: Parsing gives the equality checker mathematical expressions to compare rather than just answer text.
• Equality checker behavior and limit: Trace how equality_check decides whether an answer matches the ground truth, and state the limitation demonstrated by tuples.
  Why it matters: The check can recognize equivalent expressions, but its result is not reliable for every answer format.

Key terms: ground truth, equality check, mathematical equivalence, sympy_parser, SymPy object, implicit multiplication, equality_check, SymPy objects, tuples

3.7 Grading answers

This section builds a math-answer grader that handles multipart answers and checks it with varied test cases.

• Multipart answer splitting: Describe how split_into_parts prepares tuple-like or list-like answers for comparison by separating their comma-delimited components.
  Why it matters: Splitting lets the grader compare multipart answers component by component.
• Mathematical answer grading workflow: Trace how grade_answer compares a prediction with a ground-truth answer, including its conditions for returning True.
  Why it matters: These checks let one grader handle both simple expressions and mathematically equivalent multipart answers.
• Varied grader test cases: Describe how the section checks the grader across differently formatted answers and what the reported results establish.
  Why it matters: Varied cases help reveal whether the grader works beyond a single example—and where it may fail.

Key terms: split_into_parts, tuple-like expressions, subparts, grade_answer, prediction, ground truth, normalize, equality_check, tests, run_demos_table

3.8 Loading the evaluation dataset

This section introduces MATH-500 as the evaluation dataset, explains why it was chosen and how it is sourced, and identifies the fields needed to evaluate model answers.

• MATH-500 benchmark choice: Justify using MATH-500 to evaluate reasoning models, including its manageable size, held-out status, comparability with prior work, and suitability for challenging multistep math problems.
  Why it matters: The benchmark choice affects how practical and informative repeated model evaluations are.
• Dataset entry fields: Describe the relevant fields in a MATH-500 entry and distinguish the fields used to pose and check a problem from the worked solution.
  Why it matters: These fields show which text the model must solve and which answer its output is checked against.
• Reproducible dataset source: Explain why the chapter loads its dataset copy from the code repository and identify the optional direct-download source and dependency.
  Why it matters: A stable dataset copy helps keep evaluation results reproducible.

Key terms: MATH-500, MATH dataset, held-out evaluation set, GSM8K, problem, answer, solution, ground truth, Python dictionary, code repository

3.9 Evaluating the model

This section connects the math-answer checking components into a MATH-500 evaluation pipeline and uses its results to examine prompt sensitivity, answer errors, and the accuracy–runtime tradeoff between base and reasoning models.

• End-to-end math evaluation pipeline: Trace how a math problem becomes a graded model answer in the demo and MATH-500 evaluation functions, including why the prompt requests a boxed answer.
  Why it matters: This repeatable pipeline makes it possible to compare model and prompting changes on the same math problems.
• Prompt template sensitivity: Explain why the section tests prompt choices on more than one example and why a change in measured accuracy does not, by itself, show that answer extraction failed.
  Why it matters: Prompt comparisons are meaningful only when tested across examples and interpreted separately from extraction failures.
• Extracted answers versus correct answers: Distinguish successful answer extraction from successful math problem-solving using the demo and base-model evaluation results.
  Why it matters: Separating parsing errors from wrong answers prevents misdiagnosing a model's low accuracy.
• Reasoning accuracy and runtime tradeoff: Describe how the evaluated reasoning variant compares with the base model in accuracy and generation cost, and why the section first uses a small subset.
  Why it matters: Accuracy gains must be considered alongside evaluation time and the reliability of a small sample.

Key terms: prompt template, boxed form, mini_eval_demo, evaluate_math500_stream, MATH-500, grade_answer, base model, reasoning model variant, answer extraction, extracted

3.10 Summary

Summarizes verification-based evaluation of reasoning models, from checking free-form math answers to interpreting the evaluation results.