Reinforcement learning with verifiable rewards (RLVR) optimizes a model with rewards computed by deterministic checks rather than a learned preference model. It targets reward hacking: models can exploit approximate reward models while producing nonsensical outputs. section 2.3.5 section 7.7.3
Table 7.1 Task types mapped to reward types
|
Chapter 1 task type
|
Chapter 7 reward type
|
|
Verifiable task
|
Programmatic verifier
|
|
Open-ended task
|
Preference-trained reward model
|
Classify the task before selecting the reward: use a programmatic verifier when ground truth is objective and unambiguous; use preference-trained rewards for open-ended judgment. section 7.1.1
Apply RLVR through a short diagnostic procedure:
- Identify whether the task has programmatic ground truth and whether that ground truth captures the full objective. section 7.1.1
- Implement a verifier for exact answers, executable behavior, or structural constraints. section 7.7.3
- Return a binary reward—1 for passing and 0 for failing—and optimize the policy to produce more passing outputs. section 7.1.1 section 7.7.3
- Use preference-based signals alongside RLVR when helpfulness, faithfulness, safety, or tone cannot be fully verified. section 7.7.3
Financial research assistant
Verify that a response cites its retrieved source, conforms to the required JSON schema, and reproduces a revenue figure from the transcript. These checks convert otherwise manual judgments into verifiable reward components, while broader helpfulness and harmlessness remain preference-based. section 7.7.3
Lexical anchoring
For instruction following, BLEU can compare a student response with a high-quality synthetic reference. N-gram overlap rewards reproduction of expected facts and formatting, reducing hallucination and semantic drift, but lexical metrics remain lossy proxies for open-ended quality. section 2.3.5 section 7.1.1





