Reinforcement learning with verifiable rewards
Reinforcement learning with verifiable rewards is a training approach that uses rewards derived from objectively checkable outcomes to optimize a model or agent.
Reinforcement Learning from Human Feedback
Reinforcement learning with verifiable rewards (RLVR) is reinforcement learning in which rewards are derived from objectively checkable outcomes.
- How it is used
- Uses RLVR as a post-training stage to improve language-model performance on verifiable reasoning tasks.
- How deep it goes
- Explains verifier-based rewards and the policy-gradient loop, then summarizes practical training choices.
- Examples
- Illustrates answer matching with a prime-sum problem and situates RLVR in the Tülu 3 training recipe.
- What you can do after
- Evaluate and understand how verification signals guide RL training and how common training choices affect exploration and reasoning.
Applied Reinforcement Learning
A reinforcement-learning approach that uses objectively checkable criteria to reward model outputs.
- How it is used
- Trains language models on objectively checkable tasks by replacing learned reward models with deterministic verifier feedback.
- How deep it goes
- Explains numeric extraction, parsing, relative tolerance, and a separate format bonus for boxed answers.
- Examples
- Uses financial-report questions, a boxed numeric answer, and four correctness-and-format reward scores.
- What you can do after
- Apply it via a Python verifier to score final answers against ground truth with a relative tolerance.
Read how Applied Reinforcement Learning explains reinforcement learning with verifiable rewards →
Build a Reasoning Model (From Scratch)
A reinforcement learning approach that uses automatically checkable correctness rewards instead of human preference signals.
- How it is used
- Deploys automatically verifiable task rewards to train reasoning LLMs without human preference data or a learned reward model.
- How deep it goes
- Explains deterministic rewards, group sampling, GRPO loss, and the full training loop, but defers secure code-execution environments.
- Examples
- Uses binary math-answer verification, code compilation or execution, sampled response groups, and a PyTorch training loop with checkpoints.
- What you can do after
- Build it from scratch with a verifier, sampled rollouts, GRPO loss, optimizer updates, metrics, and model checkpoints.
LLM Evaluation and Alignment, The Foundational Ideas
Reinforcement learning from verifiable rewards is a training method that optimizes a model using rewards computed from objectively verifiable outcomes.
- How it is used
- Deploys deterministic checks to train reasoning and structured-output behavior when task outcomes have objective programmatic ground truth.
- How deep it goes
- Explains task classification, verifier construction, binary rewards, and hybrid signals, while deferring open-ended judgment to preference methods.
- Examples
- Uses exact math answers, unit-tested code, regex-constrained outputs, financial citations, JSON schemas, and BLEU-matched references.
- What you can do after
- Apply it via exact-answer comparisons, unit-test suites, regex checks, and mixed preference rewards for unverifiable objectives.
LLM Customization and Fine-Tuning
A method of fine-tuning a model with reinforcement learning using feedback on its outputs.
- How it is used
- Uses automated output grading to optimize models for tasks with programmatically verifiable outcomes.
- How deep it goes
- Explains the sampling, grading, and update loop, then compares applicability and estimated compute costs without specifying an implementation.
- Examples
- Illustrates grading with invoice extraction, SQL result sets, compliance checklists, unit tests, and simulated diagnostic conversations.
- What you can do after
- Evaluate whether a task's verifier and quality gap justify reinforcement fine-tuning's compute cost.
Read how LLM Customization and Fine-Tuning explains reinforcement learning with verifiable rewards →
Build a DeepSeek Model (From Scratch)
Reinforcement learning with verifiable rewards is reinforcement learning in which a deterministic procedure evaluates outputs and supplies the reward.
- How it is used
- Uses deterministic verification to provide scalable training rewards for tasks with objectively checkable outputs.
- How deep it goes
- Explains verifier selection, output parsing, monitoring, and scope limits, but stops short of a complete training implementation.
- Examples
- Uses arithmetic answers, unit tests, compilers, formal rules, structured final-answer markers, and held-out evaluation sets.
- What you can do after
- Apply deterministic checkers where exact correctness is available and combine them with preference signals for non-verifiable behavior.
What to know first
- reward modelA reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
- reward signalA reward signal is a value or feedback indicating the desirability, benefit, or cost associated with an action, outcome, or behavior.
- DeepSeek-R1-ZeroDeepSeek-R1-Zero is a reasoning language model trained with reinforcement learning without supervised fine-tuning as an initial warm-start stage.
- reinforcement learningReinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.
Related concepts
- reward modelcontrasts with · A reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
- verifiable rewarduses · A verifiable reward is a reward signal whose correctness can be determined by an objective criterion or procedure.
- verifieruses · A verifier is an entity or process that checks whether a claim, answer, result, or artifact satisfies specified correctness or validity criteria.
- reinforcement learningkind of · Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.
- RLHFcontrasts with · Reinforcement learning from human feedback is a model-training approach that uses human evaluations or preferences as feedback to optimize model behavior.
Questions about reinforcement learning with verifiable rewards
What is reinforcement learning with verifiable rewards?
Reinforcement learning with verifiable rewards is a training approach that uses rewards derived from objectively checkable outcomes to optimize a model or agent.
Which Manning books explain reinforcement learning with verifiable rewards?
Reinforcement Learning from Human Feedback, Applied Reinforcement Learning, Build a Reasoning Model (From Scratch), LLM Evaluation and Alignment, The Foundational Ideas, LLM Customization and Fine-Tuning and Build a DeepSeek Model (From Scratch).
What should I know before learning reinforcement learning with verifiable rewards?
Manning books that teach reinforcement learning with verifiable rewards usually introduce reward model, reward signal, DeepSeek-R1-Zero and reinforcement learning first.
What is the difference between reinforcement learning with verifiable rewards and reward model?
Reinforcement learning with verifiable rewards is a training approach that uses rewards derived from objectively checkable outcomes to optimize a model or agent. A reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
What is reinforcement learning with verifiable rewards a kind of?
Reinforcement learning with verifiable rewards is a kind of reinforcement learning. Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.
What is reinforcement learning with verifiable rewards also called?
Reinforcement learning with verifiable rewards is also called RLVR.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





