liveBook · Manning

Environment signal

As explained in Self-Improving Agents

An environment signal evaluates an agent’s output using an outcome or evidence from the external world rather than a supplied label.

An environment signal evaluates an agent’s output using evidence from the external world, not a supplied correct answer or the model’s own opinion. It is part of the signal family used to measure the gap between an output and a desired result. section 1.4.1

External evidence can include:

When available, ground-truth and environment signals are preferred because they rely on something outside the model’s own opinion. Their limitation is supply: many tasks lack a clean outcome to check. section 1.4.1

Figure 1.9 The signal family arranged by precision against availability. Ground-truth and environment signals are the most precise but the rarest; judge-based signals are available almost everywhere but are noisier, and process and reflection signals assess how the agent worked rather than only what it produced.

Compare environment signals with other signal types by their precision and availability. section 1.4.1

How Self-Improving Agents covers it

How deep it goes
Defines the signal and distinguishes it from supplied labels through concrete examples.
Examples
Uses unit-test results, tool-call outcomes, and web evidence that confirms or contradicts a claim.
What you can do after
Evaluate which feedback signal to use and favor external evidence when the task provides it.
How it connects
Places environment signals alongside ground-truth, judge-based, process, and reflection signals.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Improving Agents. The section links point to the parts of the book it draws on.