liveBook · Manning

Self-improving agent

As explained in Self-Improving Agents

An agent whose capabilities improve through changes to the shared instructions, tools, or other infrastructure around its model rather than changes to the model itself.

A production agent may handle its launch cases well, then fail on requests its developers did not anticipate. The usual fix—read a complaint, edit a prompt, redeploy, and wait—depends on scarce human attention and feedback that may be wrong. A self-improving agent runs that improvement cycle itself: it measures performance against a standard, searches for a better variant, and submits the winner for use. section 1.1.2 section 1.5

The closed loop has four steps:

  • Act: run the agent on a task and collect its output. section 1.5
  • Signal: measure the gap between the output and the standard of good. section 1.1.2 section 1.2.2
  • Search: try variants of the artifact that may close that gap. section 1.2.2
  • Apply: feed the selected change back into use, then repeat. A score alone is self-evaluation, not self-improvement. section 1.2.2 section 1.5

The model stays frozen. What changes is shared infrastructure around it: editable, versionable artifacts in the agent’s harness. These include prompts, memory, metacognitive scaffolds, and code or tools. One changed instruction can affect subsequent requests across running instances without any instance learning or any model-weight update. Shallower changes are cheaper and safer; deeper changes can be more powerful but harder to undo. section 1.1.3 section 1.5

Start with a RAG agent’s instruction

Take a retrieval-augmented generation agent whose prompt tells it how to answer from retrieved documents. Rather than changing the model or rebuilding retrieval first, make that instruction the artifact under test: evaluate its answers, search for an improved instruction, and apply the selected version. The same signal-and-search loop can later target a memory entry instead. Offline changes can be gated by deploy review. section 2.1.1 section 1.2.2 section 1.5

Figure 1.5 The three dials of the improvement loop. The signal dial sets how the gap is measured, the search dial sets how a better variant is found, and both belong to the L0 toolkit, which is reused at every layer. The artifact dial sets what is being changed, and it is the dial that the book turns as we get deeper into the book.

The three-dial diagram separates how a gap is measured, how variants are found, and which artifact changes. That separation makes the loop reusable across artifact layers. section 1.2.2

How Self-Improving Agents covers it

How deep it goes
Defines the closed loop, its three dials, artifact layers, and deployment gate; stops short of model retraining.
Examples
Uses a production-feedback failure cycle and a RAG agent whose prompt instruction is the first improvement target.
What it assumes
Assumes familiarity with agents and prompts while introducing the harness and improvement loop.
What you can do after
Build it from scratch as a prompt-improving RAG agent in Helix, while evaluating changes before they enter use.
How it connects
Connects agent harnesses, structured evaluation, search, prompts, memory, metacognition, and code tools.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Improving Agents. The section links point to the parts of the book it draws on.