liveBook · Manning

Reinforcement learning with verifiable rewards

Also called RLVR

Reinforcement learning with verifiable rewards is a training approach that uses rewards derived from objectively checkable outcomes to optimize a model or agent.

How 6 Manning books explain it

Reinforcement Learning from Human Feedback

Reinforcement learning with verifiable rewards (RLVR) is reinforcement learning in which rewards are derived from objectively checkable outcomes.

How it is used
Uses RLVR as a post-training stage to improve language-model performance on verifiable reasoning tasks.
How deep it goes
Explains verifier-based rewards and the policy-gradient loop, then summarizes practical training choices.
Examples
Illustrates answer matching with a prime-sum problem and situates RLVR in the Tülu 3 training recipe.
What you can do after
Evaluate and understand how verification signals guide RL training and how common training choices affect exploration and reasoning.

Read how Reinforcement Learning from Human Feedback explains reinforcement learning with verifiable rewards →

Applied Reinforcement Learning

A reinforcement-learning approach that uses objectively checkable criteria to reward model outputs.

How it is used
Trains language models on objectively checkable tasks by replacing learned reward models with deterministic verifier feedback.
How deep it goes
Explains numeric extraction, parsing, relative tolerance, and a separate format bonus for boxed answers.
Examples
Uses financial-report questions, a boxed numeric answer, and four correctness-and-format reward scores.
What you can do after
Apply it via a Python verifier to score final answers against ground truth with a relative tolerance.

Read how Applied Reinforcement Learning explains reinforcement learning with verifiable rewards →

Build a Reasoning Model (From Scratch)

A reinforcement learning approach that uses automatically checkable correctness rewards instead of human preference signals.

How it is used
Deploys automatically verifiable task rewards to train reasoning LLMs without human preference data or a learned reward model.
How deep it goes
Explains deterministic rewards, group sampling, GRPO loss, and the full training loop, but defers secure code-execution environments.
Examples
Uses binary math-answer verification, code compilation or execution, sampled response groups, and a PyTorch training loop with checkpoints.
What you can do after
Build it from scratch with a verifier, sampled rollouts, GRPO loss, optimizer updates, metrics, and model checkpoints.

Read how Build a Reasoning Model (From Scratch) explains reinforcement learning with verifiable rewards →

LLM Evaluation and Alignment, The Foundational Ideas

Reinforcement learning from verifiable rewards is a training method that optimizes a model using rewards computed from objectively verifiable outcomes.

How it is used
Deploys deterministic checks to train reasoning and structured-output behavior when task outcomes have objective programmatic ground truth.
How deep it goes
Explains task classification, verifier construction, binary rewards, and hybrid signals, while deferring open-ended judgment to preference methods.
Examples
Uses exact math answers, unit-tested code, regex-constrained outputs, financial citations, JSON schemas, and BLEU-matched references.
What you can do after
Apply it via exact-answer comparisons, unit-test suites, regex checks, and mixed preference rewards for unverifiable objectives.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains reinforcement learning with verifiable rewards →

LLM Customization and Fine-Tuning

A method of fine-tuning a model with reinforcement learning using feedback on its outputs.

How it is used
Uses automated output grading to optimize models for tasks with programmatically verifiable outcomes.
How deep it goes
Explains the sampling, grading, and update loop, then compares applicability and estimated compute costs without specifying an implementation.
Examples
Illustrates grading with invoice extraction, SQL result sets, compliance checklists, unit tests, and simulated diagnostic conversations.
What you can do after
Evaluate whether a task's verifier and quality gap justify reinforcement fine-tuning's compute cost.

Read how LLM Customization and Fine-Tuning explains reinforcement learning with verifiable rewards →

Build a DeepSeek Model (From Scratch)

Reinforcement learning with verifiable rewards is reinforcement learning in which a deterministic procedure evaluates outputs and supplies the reward.

How it is used
Uses deterministic verification to provide scalable training rewards for tasks with objectively checkable outputs.
How deep it goes
Explains verifier selection, output parsing, monitoring, and scope limits, but stops short of a complete training implementation.
Examples
Uses arithmetic answers, unit tests, compilers, formal rules, structured final-answer markers, and held-out evaluation sets.
What you can do after
Apply deterministic checkers where exact correctness is available and combine them with preference signals for non-verifiable behavior.

Read how Build a DeepSeek Model (From Scratch) explains reinforcement learning with verifiable rewards →

What to know first

Related concepts

Questions about reinforcement learning with verifiable rewards

What is reinforcement learning with verifiable rewards?

Reinforcement learning with verifiable rewards is a training approach that uses rewards derived from objectively checkable outcomes to optimize a model or agent.

Which Manning books explain reinforcement learning with verifiable rewards?

Reinforcement Learning from Human Feedback, Applied Reinforcement Learning, Build a Reasoning Model (From Scratch), LLM Evaluation and Alignment, The Foundational Ideas, LLM Customization and Fine-Tuning and Build a DeepSeek Model (From Scratch).

What should I know before learning reinforcement learning with verifiable rewards?

Manning books that teach reinforcement learning with verifiable rewards usually introduce reward model, reward signal, DeepSeek-R1-Zero and reinforcement learning first.

What is the difference between reinforcement learning with verifiable rewards and reward model?

Reinforcement learning with verifiable rewards is a training approach that uses rewards derived from objectively checkable outcomes to optimize a model or agent. A reward model is a learned model that assigns scores to outputs, actions, or outcomes to represent their desirability according to preferences or evaluation criteria.

What is reinforcement learning with verifiable rewards a kind of?

Reinforcement learning with verifiable rewards is a kind of reinforcement learning. Reinforcement learning is a machine-learning paradigm in which an agent learns behavior or a decision policy through interaction with an environment and evaluative feedback, such as rewards.

What is reinforcement learning with verifiable rewards also called?

Reinforcement learning with verifiable rewards is also called RLVR.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.