liveBook · Manning

Verification

As explained in Designing AI Agents

The process of checking whether a system or result satisfies specified requirements or properties.

Verification checks whether an artifact or an external change satisfies its intended requirements. For an agent, a successful tool call is not necessarily a successful action: the relevant result is the bounded change in the world, supported by evidence. section 6.0

A second model pass is not proof of correctness. Without new evidence, it can preserve an error or introduce one. Review becomes more dependable when it uses checks that are easier to ground than the original generation, such as compilation, regression tests, tool calculations, source documents, human judgment, or observed outcomes. A second model can provide a different perspective, but model identity alone does not establish independence. section 7.1.1

Choose evidence that can establish the property being checked:

  • For generated code, compile it and run a focused regression test; for financial analysis, recompute arithmetic with a tool. section 7.1.1
  • For a map overlay, inspect the result in the intended viewer. A valid image alone does not establish that the layer is usable as an overlay. section 6.0
  • For a GUI action, verify the resulting business state. A visual click is especially weak evidence of completion. section 6.3.2

Verify a payroll effect

A payroll agent can be misled by an imported note claiming that a completed payment failed. A green PAID status does not rule out a duplicate transfer. The runtime needs a durable identity connecting the authorized payment to its attempts and transfers, then must reconcile recorded attempts with provider receipts. A post-check can withhold success and trigger reconciliation or compensation, but it cannot undo a transfer merely by marking the task failed. section 6.0 section 6.5.1

Table 6.9 The three trust layers of the Guardrail Sandwich
Layer
Main question
Payroll evidence
Trusted pre-check
Is this exact effect permitted on current state?
Approved employee, current payroll version, runtime-bound arguments, commit reservation
Controlled execution
What can the running tool reach and change?
Registered payment handler, scoped credentials, bounded network and runtime
Trusted post-check
Does protected evidence satisfy the checked contract fields?
Authenticated provider receipt, one matching attempt, expected payroll transition

Compare the evidence used for admission, controlled execution, and the trusted post-check of a payment. section 6.5.1

Protect the verifier from the agent whose action it checks: the model may explain its intent, but it may not disable a required verifier or certify its own result. Verification consumes resources, so assess its reliability benefit alongside latency and cost on a defined workload. section 6.5.1 section 1.4.2

How Designing AI Agents covers it

How deep it goes
Explains evidence selection and separates admission, execution, and post-checks, including what a post-check cannot undo.
Examples
Uses regression tests, financial arithmetic, a map overlay, and payroll receipts to illustrate checks.
What it assumes
Assumes working familiarity with Python and LLM APIs, without requiring a specific agent framework.
What you can do after
Evaluate whether available evidence establishes an artifact's correctness or a requested external effect.
How it connects
Connects verification with reflection, prompt injection, idempotency, and evaluation.

Also explained in

Compare how 9 books explain verification →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.