liveBook · Manning

Reinforcement learning with verifiable rewards

As explained in Applied Reinforcement Learning

A reinforcement-learning approach that uses objectively checkable criteria to reward model outputs.

Reinforcement Learning with Verifiable Rewards (RLVR) trains a language model using a program that checks whether its answer is correct. When a task has an objective answer, this avoids learning a separate reward model to approximate human preferences. section 11.0 section 11.2

Scoring a financial answer

For a financial-report question, a Python verifier extracts the final number, parses it, and compares it with the gold answer using a small relative tolerance. Correctness earns 1.0; a boxed answer earns an additional 0.1, even when incorrect. section 11.2

The verifier assigns four possible scores:

Feedback arrives only after the model produces its final answer, not after each reasoning token. Requiring a boxed answer makes that result easy to locate; the separate format bonus distinguishes a well-formatted attempt from one that omits the expected format. section 11.1 section 11.2

Figure 11.1 FinQA financial question answering as a language-model RL environment.

See the financial question-answering task represented as a language-model reinforcement-learning environment. section 11.1

RLVR fits tasks with objectively checkable answers, especially when solving them requires multi-step reasoning. It is less suited to answers based mainly on taste or preference, where no single correct response can be verified. section 11.0 section 11.9

How Applied Reinforcement Learning covers it

How deep it goes
Explains numeric extraction, parsing, relative tolerance, and a separate format bonus for boxed answers.
Examples
Uses financial-report questions, a boxed numeric answer, and four correctness-and-format reward scores.
What it assumes
Assumes objectively checkable answers and recommends a base model that already solves some examples.
What you can do after
Apply it via a Python verifier to score final answers against ground truth with a relative tolerance.
How it connects
Connects RLVR to RLHF, reward models, GRPO, and LoRA in a model fine-tuning pipeline.

Also explained in

Compare how 6 books explain reinforcement learning with verifiable rewards →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.