A verifiable reward assigns training credit when an executable checker, test, or other objective procedure confirms that a generated response is correct. In reinforcement learning, this replaces subjective scoring for tasks such as mathematics and code, allowing rewarded behavior to become more likely. section 7.0 section 7.15
Implementing verifiable rewards requires:
- Define a task distribution with gold answers, executable tests, or another reliable correctness signal. section 7.12
- Specify a response protocol, such as placing the final answer after “Final answer:”, so the verifier can parse outputs reliably. section 7.12
- Record diagnostic outcomes—including parse failure, invalid format, timeout, and unit-test failure—in addition to the reward. section 7.12
- Normalize rewards across multiple completions for each prompt when using GRPO, while monitoring groups whose rewards are identical. section 7.12 section 7.15
Math and code tasks
A math verifier compares a parsed response with a gold answer; a code verifier executes tests. When correctness cannot be checked objectively—such as open-ended helpfulness, creative writing, subtle factuality, or safety preferences—use a learned reward model or another alignment method instead. section 7.12 section 7.13
|
Step
|
Question to ask before scaling
|
|
Task
|
Can the outcome be verified reliably?
|
|
Format
|
Can the final answer be parsed without ambiguity?
|
|
Rollout
|
Are old log probabilities and response masks stored
correctly?
|
|
Reward
|
Are parse failures and invalid outputs tracked separately?
|
|
Advantage
|
Does each prompt group have useful reward variance?
|
|
KL
|
Is the policy staying close enough to the frozen
reference?
|
|
Evaluation
|
Does held-out performance improve, not just training
reward?
|
Use the checklist table to audit task verifiability, output parsing, rollout storage, reward diagnostics, advantage variance, KL control, and held-out evaluation before scaling. section 7.12


