Verification evaluates a candidate response against a correctness criterion that can be checked definitively. In reinforcement learning with verifiable rewards (RLVR), it replaces the learned reward model used for subjective judgments with a scoring function that returns an objective signal, often reward 1 for correct and 0 for incorrect. section 7.1 section 7.5
The distinction matters when correctness is not a matter of preference. A response explaining opportunity cost must be judged for clarity, accuracy, completeness, and helpfulness, whereas a mathematics answer can be checked against a known value. Verification therefore makes tasks such as math, code, and some science problems suitable for direct correctness-based reward. section 7.1 section 7.5
A verification pipeline usually has three operational parts:
- Define the expected criterion, such as an exact answer or behavior on known inputs. section 7.1
- Extract or execute the relevant part of the response. Mathematical answers may use boxed notation, “The answer is:”, special tokens, or delimiters to make extraction reliable. section 7.1
- Convert the result into a reward. A simple gate gives reward 1 only when every required assertion passes, and 0 otherwise. section 7.1
Exact-answer verification
For a prompt asking for the sum of primes below 20, the expected result is 77. The response can expose its final result as \boxed{77}; a regular expression extracts that value, and extracted_answer == 77 receives reward 1. This check scores whether the submitted answer matches the criterion rather than requiring a learned judgment of prose quality. section 7.1
Code verification with unit tests
For a generated Fibonacci function, assertions such as fib(0) == 0, fib(1) == 1, and fib(10) == 55 execute the candidate implementation against expected input-output pairs. If all assertions pass, the solution receives reward 1. Unit tests are a natural verifier because they test behavior directly. section 7.1
Within RLVR, verification supplies the signal that selects useful sampled answers. The training loop samples multiple answers, takes gradient steps toward correct ones, and repeats. This setup is closely associated with reasoning training and inference-time scaling: models are trained to produce longer reasoning chains correlated with improved performance on verifiable tasks. section 7.5
Verification does not make every task objective. It is best suited to settings with assessable correctness; subjective qualities instead require learned preferences or another way to judge them. Practical RLVR recipes may also add format and language-consistency rewards, filter by task difficulty, relax KL penalties for exploration, and schedule response length to reduce overthinking. section 7.1 section 7.5
