liveBook · Manning

Verifier

As explained in Designing AI Agents

A verifier is a component that checks whether an answer or result satisfies relevant criteria.

A verifier checks whether an agent’s result satisfies relevant criteria. In a recovery loop, it turns a failure into a concrete signal: after a bounded repair, the agent can check whether the change worked. Verification is limited to conditions the verifier can observe and judge. section 7.3.1

Keep the verifier, acceptance criteria, and protected tests read-only during recovery. A process proposing a repair should not be able to rewrite the evidence used to approve it. The loop should verify each mutation, preserve a rollback point, stop when progress stalls, and hand off failures outside its authority. section 7.3.1

Repairing a tool invocation

A tool call fails because a newer tool version removed an argument. Inside a sandbox, the agent inspects the current contract, revises the invocation, and runs the check again. If the failure instead requires authoritative domain knowledge the system lacks, another retry cannot supply it; the agent should request that knowledge or hand off the task. section 7.3.1

Figure 7.2 The Self-Heal Loop. The agent captures a baseline, diagnoses a failure, checks repair authority, applies a change atomically, verifies it, and either recovers, retries, rolls back, or hands off.

Follow the recovery loop from baseline and authority check through verification, retry, rollback, or handoff. section 7.3.1

Verification is distinct from admission and containment. A sandboxed command’s successful exit does not prove that the intended test ran or that a remote outcome occurred. In a code-repair flow, a surrounding service can rerun applicable verifiers before opening a pull request and refuse publication; even then, correctness is established only within what those checks observe. section 6.5.2

A separate verifier also has a cost. Add one to a reasoning flow when its measured improvement justifies the extra call. Repeated checks and repairs can add latency, token use, and state complexity; agreement among sampled answers is evidence of stability, not proof of truth. section 5.2.5 section 7.3.1

How Designing AI Agents covers it

How deep it goes
Explains protected verification boundaries and recovery steps, but limits assurance to conditions the checks can observe and judge.
Examples
Uses a removed tool argument and a code-repair publication gate to illustrate checks after agent changes.
What you can do after
Evaluate where verification supports recovery or publication, where it falls short, and whether an extra verifier call justifies its cost.
How it connects
Connects verification to sandboxing, acceptance criteria, rollback, guardrails, and self-consistency.

Also explained in

Compare how 9 books explain verifier →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.