liveBook · Manning

Proof gate

As explained in Self-Improving Agents

A binary verifier that compares an agent’s claim with a computable truth and classifies it as proven or unproven.

A proof gate checks an agent’s claim against a truth computed independently. It returns a binary verdict: accept if the claim matches, reject if it does not. When the truth is computable, the verdict needs no sample, judge, or confidence estimate. section 3.1.2

A sum and a letter count

For “1 + 1 + 1,” the agent answers 3, which matches the computed truth and is accepted. Asked how many r’s appear in “strawberry fields are forever and ever,” it answers 8; Python’s `LINE.count("r")` returns 7, so the gate rejects the claim. A five-word summary can be checked for length, but not for a single correct meaning. section 3.1.2 section 3.1.2

Use a proof gate when a checker can derive the truth:

  • Write a checker that computes the answer independently, then compare it with the agent’s claim. section 3.1.2
  • Treat the result as exact: the claim is proven or unproven, not partly correct. If only part of an answer is computable, verify that part and use another signal for the rest. section 3.1.2
  • To practice, create questions with fully computable, partly checkable, and unchecked answers; build available checkers and predict their verdicts. section 3.5
Figure 3.2 The proof gate and the measurement that replaces it. When a claim has a computable truth, the gate derives the truth itself and compares, so the verdict is certain: the agent's 3 for the sum is accepted, and its 8 for the letter count is rejected because LINE.count("r") computes 7. When no checker exists, as with a five-word summary, the gate has nothing to compute, and verification degrades to measurement. Almost everything an agent produces lives on that lower path.

The diagram contrasts certain computed verdicts with answers that lack a checker and must be evaluated another way. section 3.1.2

How Self-Improving Agents covers it

How deep it goes
Defines a binary verifier and works through arithmetic and string-count checks, stopping where answers lack a computable truth.
Examples
Uses a sum, counting r’s in a line, and a five-word summary to contrast computable truth with semantic judgment.
What you can do after
Build proof gates for computable questions, predict verdicts, and identify when another signal must replace exact verification.
How it connects
Connects exact verification to measurement for unchecked semantics and to smooth scoring as an alternative exercise.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Improving Agents. The section links point to the parts of the book it draws on.