A production agent may handle its launch cases well, then fail on requests its developers did not anticipate. The usual fix—read a complaint, edit a prompt, redeploy, and wait—depends on scarce human attention and feedback that may be wrong. A self-improving agent runs that improvement cycle itself: it measures performance against a standard, searches for a better variant, and submits the winner for use. section 1.1.2 section 1.5
The closed loop has four steps:
- Act: run the agent on a task and collect its output. section 1.5
- Signal: measure the gap between the output and the standard of good. section 1.1.2 section 1.2.2
- Search: try variants of the artifact that may close that gap. section 1.2.2
- Apply: feed the selected change back into use, then repeat. A score alone is self-evaluation, not self-improvement. section 1.2.2 section 1.5
The model stays frozen. What changes is shared infrastructure around it: editable, versionable artifacts in the agent’s harness. These include prompts, memory, metacognitive scaffolds, and code or tools. One changed instruction can affect subsequent requests across running instances without any instance learning or any model-weight update. Shallower changes are cheaper and safer; deeper changes can be more powerful but harder to undo. section 1.1.3 section 1.5
Start with a RAG agent’s instruction
Take a retrieval-augmented generation agent whose prompt tells it how to answer from retrieved documents. Rather than changing the model or rebuilding retrieval first, make that instruction the artifact under test: evaluate its answers, search for an improved instruction, and apply the selected version. The same signal-and-search loop can later target a memory entry instead. Offline changes can be gated by deploy review. section 2.1.1 section 1.2.2 section 1.5
Figure 1.5 The three dials of the improvement loop. The signal dial sets how the gap is measured, the search dial sets how a better variant is found, and both belong to the L0 toolkit, which is reused at every layer. The artifact dial sets what is being changed, and it is the dial that the book turns as we get deeper into the book.

The three-dial diagram separates how a gap is measured, how variants are found, and which artifact changes. That separation makes the loop reusable across artifact layers. section 1.2.2
