chapter one

1 Foundations of self-improving agents

 

This chapter covers

  • What a self-improving agent is, why the model never changes, and what changes instead
  • The improvement loop and its three dials: signal, search, harness-artifact
  • The four kinds of things the loop can improve: prompts, memory, reasoning, and code
  • The signal and search families behind the loop

Not long after the first capable LLM agents appeared, the people building them ran into the same wall, one after another. Getting an agent working well enough to perform a discrete set of use cases, turned out to be the quick part, often a week of prompting and a few tools wired up. Keeping that agent running once real users got hold of it was the part that dragged on for months.

The reasons were simple; an agent developed against a set of use cases will fail in production when it faces every case your users invent, and the gap between those two shows up as additional cases of things the agent gets wrong. So, you do what the job seems to ask: you read the latest complaint, adjust a prompt by hand or with an LLM, redeploy, and wait for the next one. The problem isn’t that you missed everything a user may attempt, it's that you can rarely predict the bounds of an AI agent without some form of iteration.

1.1 What a self-improving agent is

1.1.1 Defining the agent and the scaffolding

1.1.2 The self-improving agent

1.1.3 Why the model never changes

1.2 The improvement loop

1.2.1 The loop, walked through one task

1.2.2 The three dials

1.2.3 Online and offline: where the loop runs

1.3 The four kinds of things the loop can improve

1.3.1 What an artifact is

1.3.2 The four layers and the toolkit that runs them

1.4 The signal and search families behind the loop

1.4.1 The signal family

1.4.2 The search family

1.5 Summary