liveBook · Manning

Signal ceiling

As explained in Self-Improving Agents

The limitation that any individual evaluative signal captures only part of the gap between an agent’s output and the desired result.

A signal ceiling is the limit of any single evaluation signal: it captures only part of the gap between an agent’s output and the desired result. A score gives magnitude without cause; a critique gives possible causes without magnitude, and may still miss failures. section 3.3.2 section 3.6

One run, two partial signals

A revenue retrieval run scores 0.63 against ground truth, which identifies no cause. A reflection names three missed rules—excluding cancelled and returned orders, internal channels, and pending payments—but misses the refund column. section 3.3.2

Figure 3.9 The signal ceiling. One genesis run is measured twice: ground truth fills `score` with 0.63 and says nothing about cause, the symptom, while reflection fills `feedback` with the missed filters and says nothing about magnitude, the diagnosis. Even the diagnosis is partial, naming three of the four missed rules and skipping the refund column. Each signal measures less than the whole gap, so turning the dial changes what we see, not how much of it.

Compare the score and critique for the same run, including the failure the critique overlooks. section 3.3.2

How Self-Improving Agents covers it

How deep it goes
Contrasts scalar scores and written critiques through a worked run, then generalizes their shared incompleteness.
Examples
Uses a revenue retrieval run scored 0.63 whose critique names three missed rules but omits refunds.
What you can do after
Apply the insight by pairing a score for magnitude with a critique for direction, giving a proposer richer feedback than either alone.
How it connects
Connects ground truth, reflection, composite signals, and the measure–propose–select–record improvement loop.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Improving Agents. The section links point to the parts of the book it draws on.