liveBook · Manning

Composite signal

As explained in Self-Improving Agents

A feedback signal that combines multiple kinds of evaluation information, such as a score and a critique.

A composite signal combines a score with a critique, giving search both the magnitude of a gap and direction for proposing changes. For example, ground truth paired with reflection can be more useful to search than a clean proof alone: the proof may be more certain, but offer little guidance about what to change. Neither part captures the whole gap. section 3.4 section 3.6

Pack the score and critique

Diagnose a genesis run by predicting which broken rules its critique will name, then compare that prediction with the run. Map each complaint to a rule, note any unnamed rule, and combine the smooth score and critique into one GapMeasurement. Record what each contributes and any blind spot inherited from the critique. section 3.5

How Self-Improving Agents covers it

How deep it goes
Pairs score magnitude with critique direction, while recognizing that each captures only part of the gap.
Examples
Uses a genesis run to map critique onto broken rules and pack a smooth score with critique.
What you can do after
Diagnose a run by comparing predicted and observed critique, then combine its score and critique in a GapMeasurement.
How it connects
Connects search with ground truth, reflection, proof, and the improvement loop.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Improving Agents. The section links point to the parts of the book it draws on.