liveBook · Manning

GapMeasurement

As explained in Self-Improving Agents

A structured evaluation result that records an assessment of the gap between a candidate output and a desired standard, along with supporting feedback.

GapMeasurement is a shared result shape for measuring the difference between an agent’s output and a chosen standard. It keeps downstream search and archives independent of which method produced the signal. section 2.2.2 section 3.1.4

The result can carry:

  • A score, preference between two answers, or feedback sentence; each can be left empty when a signal does not supply it. section 3.1.4
  • Confidence and bookkeeping fields for signal ID, cost, and metadata. The proven flag marks a formal proof, not an ordinary ground-truth check. section 3.1.4

Different signals, one result

Ground truth can fill the score; an LLM judge can fill the preference; reflection can fill feedback. A formal proof sets a binary score and proven=true. Downstream code reads populated fields without branching on signal type. section 3.1.4

Figure 3.4 The GapMeasurement union. Every signal returns the same object: a proof fills the score with 1.0 or 0.0 and sets proven, ground truth fills the score, the chapter-2 judge fills the preference, and a reflection fills the feedback, while confidence and the bookkeeping fields travel with every measurement. Search and the archive read whichever field is filled, so the loop never branches on which signal ran.

Shows how proof, ground truth, judging, and reflection converge on the shared result shape. section 3.1.4

How Self-Improving Agents covers it

How deep it goes
Defines optional measurement fields, confidence, proof status, and bookkeeping, stopping at the shared result shape.
Examples
Maps ground truth to score, judges to preference, reflection to feedback, and proofs to binary scores with proven=true.
What it assumes
Assumes a chosen quality standard and measurement method; leaves their selection outside the shared result shape.
What you can do after
Understand how to pass heterogeneous evaluation signals downstream through one result, reading whichever fields each signal populates.
How it connects
Connects signal measurement to ground truth, LLM-as-Judge, reflection, formal proof, search, and artifact archives.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Improving Agents. The section links point to the parts of the book it draws on.