Verifiable reward

As explained in Build a Reasoning Model (From Scratch)

A reward signal that can be automatically checked against a correct answer or objective criterion.

Verifiable rewards turn correctness into a deterministic training signal. A verifier checks a generated answer against a reference solution, then assigns a reward such as 1 for correct or 0 for incorrect. section 3.0 section 6.1.2

A verifiable-reward workflow:

• Generate a response for a problem. section 6.1.2
• Run a deterministic verifier against the correct solution and required output format. section 6.13
• Use the resulting reward directly to update the model, without a learned reward model or human annotation. section 6.1.2

Figure 6.6 Overview of RLVR. The LLM generates a response that is evaluated by a deterministic verifier, such as the math verifier from chapter 3, which assigns a correctness label that’s used as a reward signal within an RL objective to update the model.

Follow the response-to-verifier-to-reward-to-update flow used by RLVR. section 6.1.2

How Build a Reasoning Model (From Scratch) covers it

How deep it goes

Defines binary verification and the training loop, while deferring secure code-execution environments.

Examples

Uses calculator-like math checking, reference answers, binary correctness labels, and required boxed output.

What it assumes

Assumes generated responses, correct solutions, and a reinforcement-learning update loop.

What you can do after

Build a deterministic verifier that converts correct, extractable answers into training rewards.

How it connects

Connects deterministic verification to RLVR, RLHF, GRPO, and policy-gradient optimization.

liveBookmanning.com

Read Build a Reasoning Model (From Scratch) on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackA verifiable reward is a reward signal whose correctness can be checked objectively by a procedure or criterion.

Compare how 2 books explain verifiable reward →

Related concepts

• reinforcement learning with verifiable rewardsused by · Reinforcement learning with verifiable rewards is a training approach that optimizes a model using rewards derived from objectively checkable outcomes.
• Group Relative Policy Optimizationused by · Group Relative Policy Optimization is a reinforcement-learning algorithm that updates a policy using relative rewards or advantages estimated from groups of sampled outputs or actions.
• reward hackingcontrasts with · Reward hacking is behavior that exploits weaknesses or mismatches in a reward or evaluation specification to obtain a high measured score without achieving the intended objective.
• reinforcement learning with human feedbackcontrasts with · Reinforcement learning with human feedback is a model-training approach that uses human evaluations or preferences to guide optimization of model behavior.
• reinforcement learningused by · Reinforcement learning is a machine-learning paradigm in which an agent learns actions or a policy through interaction with an environment and evaluative feedback, such as rewards.

This explainer was generated by Manning's knowledge graph from the text of Build a Reasoning Model (From Scratch). The section links point to the parts of the book it draws on.