liveBook · Manning

Verifier

As explained in Applied Reinforcement Learning

A verifier is a procedure that checks a model's answer against a specified correctness criterion or reference answer.

A verifier checks a model’s answer against a known correctness criterion. When a task has a gold answer, it can parse the model’s final response and compare it with that reference, providing a deterministic reward instead of relying on a learned reward model to estimate preference. section 11.0 section 11.1

Checking a financial answer

For a financial question, the model generates a complete response and places its final answer in a LaTeX box. The verifier extracts that answer and compares it with the gold answer; the reward arrives only at the end, so useful reasoning can still receive zero if the answer is wrong or cannot be parsed. section 11.1

The check depends on what counts as verifiable in the task:

  • Compare a financial answer with its reference value. section 11.1
  • Run generated code against a test suite and use pass or fail as feedback. section 11.8.4
  • Use a molecular docking simulation to score a candidate molecule’s binding affinity. section 11.8.3
Figure 11.1 FinQA financial question answering as a language-model RL environment.

See how the verifier supplies a final reward in the financial question-answering task. section 11.1

How Applied Reinforcement Learning covers it

How deep it goes
Parses a final answer and compares it with a reference, but provides no token-by-token feedback during generation.
Examples
Uses FinQA gold answers and code test-suite results as concrete verification targets.
What it assumes
Assumes the task has a ground-truth answer or objective test rather than a subjective correctness judgment.
What you can do after
Build it from scratch as a function that parses a final answer and checks it against a gold answer.
How it connects
Connects reinforcement learning with verifiable rewards to reward design and domain-specific measurement.

Also explained in

Compare how 9 books explain verifier →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.