Reinforcement learning with verifiable rewards (RLVR) uses a deterministic checker instead of a learned reward model. The checker evaluates outputs against objective criteria, such as a known mathematical answer, unit tests, a compiler, or a formal rule. section 7.5
A reliable RLVR pipeline must treat verification as part of the training problem:
- Define a strict response protocol, such as placing the answer after “Final answer:”, so the verifier extracts the intended result rather than prose or multiple candidates. section 7.10.1
- Extract runnable code deterministically by identifying the expected file or function before executing tests; brittle extraction can create misleading rewards. section 7.10.1
- Monitor pass rate, response length, KL divergence, invalid-format rate, duplicate outputs, reward variance, and held-out performance rather than average reward alone. section 7.10.2
Match the checker to the task
For an arithmetic problem, compare the final number with the known answer. For a programming problem, run unit tests or a compiler. For helpfulness, harmlessness, creative writing, or nuanced dialogue, use preference data, human evaluation, or a learned reward model because no single deterministic answer exists. section 7.5 section 7.7.5
Figure 7.10 RLVR can replace learned reward models for objectively verifiable tasks such as math and code; open-ended assistant behavior still requires preference signals.

Use the figure to distinguish tasks where exact checkers can replace learned reward models from open-ended behavior that still requires preference signals. section 7.5





