Reinforcement learning with verifiable rewards
Reinforcement learning with verifiable rewards is a training approach that optimizes a model using rewards derived from objectively checkable outcomes.
Reinforcement Learning from Human Feedback
Reinforcement learning with verifiable rewards (RLVR) is reinforcement learning in which rewards are derived from objectively checkable outcomes.
- How it is used
- Boosts reasoning, coding, mathematics, and other skills whose outputs can receive definitive correctness feedback.
- How deep it goes
- Defines verifier-based rewards, maps the sampling-and-policy-gradient loop, and lists training practices without deriving the underlying algorithms.
- Examples
- Uses prime-sum answer extraction, unit tests, code execution, theorem proving, and Tülu 3 and DeepSeek R1 training recipes.
- What you can do after
- Apply it via verifier-backed RL tooling to reinforce correct outputs on objectively checkable tasks.
Build a Reasoning Model (From Scratch)
A reinforcement learning approach that uses automatically checkable correctness rewards instead of human preference signals.
- How it is used
- Deploys automatically verifiable task rewards to train reasoning LLMs without human preference data or a learned reward model.
- How deep it goes
- Explains deterministic rewards, group sampling, GRPO loss, and the full training loop, but defers secure code-execution environments.
- Examples
- Uses binary math-answer verification, code compilation or execution, sampled response groups, and a PyTorch training loop with checkpoints.
- What you can do after
- Build it from scratch with a verifier, sampled rollouts, GRPO loss, optimizer updates, metrics, and model checkpoints.
Related concepts
- reward modelcontrasts with · A reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.
- verifiable rewarduses · A verifiable reward is a reward signal whose correctness can be determined by an objective criterion or procedure.
- reinforcement learningkind of · Reinforcement learning is a machine-learning paradigm in which an agent learns actions or a policy through interaction with an environment and evaluative feedback, such as rewards.
- verifieruses · A verifier is a person, component, or procedure that checks whether a claim, result, artifact, or credential satisfies specified correctness or validity criteria.
- reinforcement learning with human feedbackcontrasts with · Reinforcement learning with human feedback is a model-training approach that uses human evaluations or preferences to guide optimization of model behavior.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.