Chapter 1: Foundations of self-improving agents
The introduction explains why production agents need ongoing improvement and presents a self-run improvement loop that does not retrain the underlying model.
After this chapter you will be able to
- Production use-case gap. Explain why an agent that works on its development use cases may still need repeated improvement after real users begin using it.
- Self-run improvement loop. Describe the proposed replacement for manually responding to each new agent failure, including what the loop does and what it does not change.
- Agent and harness boundary. Explain how the harness supports an agent and how a model running within a configured harness forms a task-performing agent.
- Self-improving agent cycle. Explain what makes an agent self-improving: it measures outputs against a standard, searches for a better version of the agent or its harness, and submits the winner without a human driving each cycle or changing the underlying model.
- Repeatable evaluation signal. Explain why user complaints are an unreliable basis for improvement and how a structured, repeatable evaluation changes the signal, while humans still approve major changes and deployment remains gated.
- Frozen model and shared artifacts. Explain where improvement lives when the model stays frozen, and describe how changing a shared prompt can affect many agent instances without any instance learning.
- Basic improvement loop. Describe how an agent produces a result, a weak or incorrect result can signal a search for improvement, and a found improvement is applied back to the agent before the loop repeats.
- Four-step improvement loop. Trace the support-agent loop from answering a question through measuring a gap, searching for a better prompt, and applying the winner before the next task.
- Judge-based gap signal. Explain how the LLM-as-Judge identifies a useful gap in an answer that is factually correct.
- Prompt-variant selection. Describe how search uses the judge's signal to select a prompt change without changing the model or requiring labelled data.
- Three improvement loop dials. Describe the roles of signal, search, and artifact: the signal measures a gap, the search finds a better variant, and the artifact is what gets changed.
- Independent dial settings. Explain how one dial can change while the other two stay fixed, using the section's switch from a prompt artifact to a memory entry as an example.
- Closed improvement loop. Distinguish self-improvement from self-evaluation by explaining that a score must feed a search and the resulting change must be put back into use.
- Deployment gate and loop. Distinguish offline from online improvement while explaining what stays the same. Offline changes are evaluated and pass through a human-reviewed deployment gate; online changes can reach the running agent on the next request.
- Online improvement limits. Explain why removing the human deployment gate restricts online changes and their feedback. Live changes should be cheap and reversible, typically prompts or memory, while direct user feedback provides a narrow signal.
- Artifact replacement principle. Explain that improving an agent, from the artifact’s point of view, means producing a better version of a piece and replacing the old one; the prompt changed in section 1.2 is an example of a text artifact.
- Artifact criteria and examples. Explain what makes something an artifact and give examples beyond prompts. Describe how the improvement loop uses candidate versions.
- Artifact boundaries and exclusions. Distinguish tracked artifacts from model internals and external-provider foundations, using the section's reasons for excluding them.
- Context and memory artifacts. Distinguish L1 text or context artifacts from L2 memory artifacts, and describe what improving each changes. Give examples of L1 context beyond prompts and of lessons stored across tasks in L2.
- Metacognition and harness code. Distinguish L3 routines that govern how the agent reasons from L4 code, tools, and orchestration that run the harness. Describe an example of a change at each layer.
- Depth and change risk. Explain why moving from L1 toward L4 increases both the potential power and the danger of an artifact change, and relate that trade-off to the preference for shallow changes in online loops.
- Layer-zero improvement toolkit. Describe what layer zero does across all four artifact layers and distinguish the Helix example from a framework readers must adopt.
- Signal and search roles. Distinguish the signal, which determines how far the improvement loop can go, from search, which acts once a gap can be measured.
- Signal choice tradeoff. Explain what an improvement signal measures and why the practical choice is the most precise signal available for the task.
- Signal quality ceiling. Explain why self-judgment alone eventually stops driving reliable improvement, and contrast it with the impractical precision of formal proof.
- Candidate search and exploration. Explain how search generates artifact variants and uses a signal to select among them, contrasting hill-climbing with pairwise and population methods.
- Search effort and reach. Explain the cost-versus-improvement-reach tradeoff and state the section's rule for choosing search effort.
- Search portability across artifacts. Explain why a named search technique need not be confined to the artifact layer used in its original demonstration, and identify the qualification for RL searches.
Section by section
Introduction
The introduction explains why production agents need ongoing improvement and presents a self-run improvement loop that does not retrain the underlying model.
- Production use-case gap: Explain why an agent that works on its development use cases may still need repeated improvement after real users begin using it.
Why it matters: It explains why getting an agent working initially is not the end of the engineering work. - Self-run improvement loop: Describe the proposed replacement for manually responding to each new agent failure, including what the loop does and what it does not change.
Why it matters: It states the book's central approach to improving agents without changing their underlying model.
Key terms: use cases, production, iteration, improvement loop, underlying model
1.1 What a self-improving agent is
This section previews a review of the agent and its surrounding harness, followed by an explanation of self-improvement.
1.1.1 Defining the agent and the scaffolding
Defines an agent in relation to its supporting harness and explains why improving an agent may include improving that harness.
- Agent and harness boundary: Explain how the harness supports an agent and how a model running within a configured harness forms a task-performing agent.
Why it matters: This boundary clarifies what is doing the task and what software supports it. - Harness improvement in agent work: Explain why the book may treat harness changes as part of improving an agent and describe how it frames that work around measurable agent tasks.
Why it matters: It keeps the focus on better task outcomes even when the change is made to supporting software.
Key terms: agent, harness, scaffolding, model, self-improvement, agent tasks
1.1.2 The self-improving agent
This section contrasts manual, feedback-driven agent fixes with a self-improving agent that evaluates outputs, searches for improvements to its agent and harness, and proposes changes while humans retain deployment oversight.
- Self-improving agent cycle: Explain what makes an agent self-improving: it measures outputs against a standard, searches for a better version of the agent or its harness, and submits the winner without a human driving each cycle or changing the underlying model.
Why it matters: This distinction identifies what is being improved and what remains fixed. - Manual improvement workflow: Trace the usual manual cycle from user complaint through engineer review, prompt or context edit, redeployment, and waiting for more feedback; describe why this makes improvement slow and fixes hard to roll back.
Why it matters: The manual workflow shows the delay and uncertainty the self-improving cycle is meant to reduce. - Repeatable evaluation signal: Explain why user complaints are an unreliable basis for improvement and how a structured, repeatable evaluation changes the signal, while humans still approve major changes and deployment remains gated.
Why it matters: A reliable signal helps prevent the agent from optimizing for a misleading complaint. - Improvable harness artifacts: Describe how the same improvement pattern applies across RAG, task, batch task, and coding agents by identifying examples of prompts, memory, planning scaffolds, or tools that can be improved.
Why it matters: Recognizing artifacts in different agents makes the improvement idea applicable beyond prompt edits.
Key terms: self-improving agent, harness, underlying model, user feedback, prompt/context, redeploy, unversioned, signal, structured, repeatable evaluation, deploy gate
1.1.3 Why the model never changes
The section explains why a self-improving agent improves its shared infrastructure and artifacts rather than changing the frozen model.
- Individual and infrastructure intelligence: Explain the analogy between human progress and agent improvement: capabilities can grow when shared tools and processes improve, even if individual brains or models do not.
Why it matters: The analogy makes it easier to see how an agent can improve without altering its model. - Frozen model and shared artifacts: Explain where improvement lives when the model stays frozen, and describe how changing a shared prompt can affect many agent instances without any instance learning.
Why it matters: It identifies what can actually change in the book’s approach to self-improvement.
Key terms: infrastructure intelligence, individual intelligence, collective intelligence, frozen model, agent context instructions, harness, artifacts
1.2 The improvement loop
Introduces the recurring loop in which an agent’s measured result can prompt a search for an improvement that is applied back to the agent.
- Basic improvement loop: Describe how an agent produces a result, a weak or incorrect result can signal a search for improvement, and a found improvement is applied back to the agent before the loop repeats.
Why it matters: This cycle is the basic pattern behind the later examples of agent improvement.
Key terms: agentic self-improvement, loop, search for improvement
1.2.1 The loop, walked through one task
A support-agent example shows how an offline improvement loop judges an answer, selects a better prompt, and deploys it without changing the model.
- Four-step improvement loop: Trace the support-agent loop from answering a question through measuring a gap, searching for a better prompt, and applying the winner before the next task.
Why it matters: This sequence shows how an agent's performance can improve across tasks rather than during a single answer. - Judge-based gap signal: Explain how the LLM-as-Judge identifies a useful gap in an answer that is factually correct.
Why it matters: A correct answer can still need improvement, and the judge's reason gives the search a target. - Prompt-variant selection: Describe how search uses the judge's signal to select a prompt change without changing the model or requiring labelled data.
Why it matters: The example shows that a small, tested prompt change can improve the agent without retraining its model.
Key terms: act, signal, search, apply, deploy gate, offline, LLM-as-Judge, rubric, gap, prompt variants
1.2.2 The three dials
This section identifies three independent dials of the improvement loop and explains why a score alone does not constitute self-improvement.
- Three improvement loop dials: Describe the roles of signal, search, and artifact: the signal measures a gap, the search finds a better variant, and the artifact is what gets changed.
Why it matters: Separating these roles makes it easier to describe what an improvement attempt changes. - Independent dial settings: Explain how one dial can change while the other two stay fixed, using the section's switch from a prompt artifact to a memory entry as an example.
Why it matters: The same measurement and search approach can be reused when the thing being improved changes. - Closed improvement loop: Distinguish self-improvement from self-evaluation by explaining that a score must feed a search and the resulting change must be put back into use.
Why it matters: A dashboard showing scores does not by itself show that the agent is improving.
Key terms: signal, search, artifact, gap, variant, three dials, artifact dial, prompt, memory, self-evaluation
1.2.3 Online and offline: where the loop runs
This section contrasts online and offline versions of the same improvement loop, explaining how the deployment gate affects safety, feedback, and the kinds of changes each version can make.
- Deployment gate and loop: Distinguish offline from online improvement while explaining what stays the same. Offline changes are evaluated and pass through a human-reviewed deployment gate; online changes can reach the running agent on the next request.
Why it matters: The gate determines when an agent-proposed improvement can affect users. - Online improvement limits: Explain why removing the human deployment gate restricts online changes and their feedback. Live changes should be cheap and reversible, typically prompts or memory, while direct user feedback provides a narrow signal.
Why it matters: Fast live adaptation comes with tighter safety and feedback limits. - Complementary improvement cadences: Describe how an agent can use offline and online improvement together. Offline loops handle larger capability improvements between deployments through evaluation and gated rollout, while online loops make small, fast prompt and memory adaptations.
Why it matters: An agent need not choose between careful gated improvement and rapid small adjustments.
Key terms: offline, online, deployment gate, signal, search, artifacts, prompts, memory, scaffolds, code
1.3 The four kinds of things the loop can improve
This section introduces artifacts as the pieces an improvement loop replaces with better versions.
- Artifact replacement principle: Explain that improving an agent, from the artifact’s point of view, means producing a better version of a piece and replacing the old one; the prompt changed in section 1.2 is an example of a text artifact.
Why it matters: It gives you a simple way to describe what changes when an agent improves.
Key terms: artifact, text artifact (context)
1.3.1 What an artifact is
Defines the editable, tracked artifacts an improvement loop can change, distinguishes them from excluded internals, and explains why artifact layers can overlap.
- Artifact criteria and examples: Explain what makes something an artifact and give examples beyond prompts. Describe how the improvement loop uses candidate versions.
Why it matters: This definition identifies what the improvement loop can safely test and change. - Artifact boundaries and exclusions: Distinguish tracked artifacts from model internals and external-provider foundations, using the section's reasons for excluding them.
Why it matters: The boundary keeps self-improvement focused on parts the loop can actually track and revise. - Overlapping artifact layers: Explain why artifact layers are building blocks rather than sealed categories, using the code-layer search example.
Why it matters: It prevents treating a layer choice as a strict limit on which artifacts a search may change.
Key terms: artifact, harness, candidate version, rollback-able, LLM weights, activations, internal states, external loop, layers, code layer
1.3.2 The four layers and the toolkit that runs them
This section classifies editable agent artifacts by how deeply they change the harness, explains how the layers compose and differ in risk, and distinguishes them from the layer-zero toolkit that runs improvement.
- Context and memory artifacts: Distinguish L1 text or context artifacts from L2 memory artifacts, and describe what improving each changes. Give examples of L1 context beyond prompts and of lessons stored across tasks in L2.
Why it matters: This distinction identifies whether an improvement changes what the model is told now or what the agent retains for future tasks. - Metacognition and harness code: Distinguish L3 routines that govern how the agent reasons from L4 code, tools, and orchestration that run the harness. Describe an example of a change at each layer.
Why it matters: These layers separate changes to an agent's reasoning routine from changes to the machinery running it. - Depth and change risk: Explain why moving from L1 toward L4 increases both the potential power and the danger of an artifact change, and relate that trade-off to the preference for shallow changes in online loops.
Why it matters: The depth of a proposed edit helps determine how cautiously it should be deployed. - Composition across artifact layers: Explain how an artifact at one layer can contain or use components from another layer, using the section's skill-file or metacognitive-scaffold example.
Why it matters: Composition explains why changing one artifact can matter inside a larger scaffold or memory-backed capability. - Layer-zero improvement toolkit: Describe what layer zero does across all four artifact layers and distinguish the Helix example from a framework readers must adopt.
Why it matters: The same improvement plumbing can operate on a prompt or a code artifact without being one of the artifacts it changes.
Key terms: L1, text or context layer, tool descriptions, skill files, L2, memory layer, L3, metacognitive layer, scaffold, L4
1.4 The signal and search families behind the loop
Introduces the distinct roles of signal and search in the improvement loop.
- Signal and search roles: Distinguish the signal, which determines how far the improvement loop can go, from search, which acts once a gap can be measured.
Why it matters: The distinction explains why finding a gap and acting on it are separate parts of improvement.
Key terms: signal, search, gap, L0 toolkit
1.4.1 The signal family
This section compares three families of improvement signals, explains the tradeoff between their precision and availability, and shows why signal quality limits improvement.
- Signal choice tradeoff: Explain what an improvement signal measures and why the practical choice is the most precise signal available for the task.
Why it matters: The loop can only use feedback you can obtain, and that feedback must distinguish better work from worse work. - Grounded external signals: Distinguish ground-truth signals from environment signals and explain both their strength and their scarcity.
Why it matters: External checks are worth using when possible, but many agent tasks cannot supply enough of them. - Model judge signals: Describe absolute, pairwise, and contrastive judging and explain the availability and precision tradeoff of model judges.
Why it matters: Model judges make feedback widely available, but their verdicts should not be treated as ground truth. - Process and reflection feedback: Distinguish feedback on an agent’s steps and trajectory from feedback on its final output.
Why it matters: A good-looking answer can conceal a weak route to that answer. - Signal quality ceiling: Explain why self-judgment alone eventually stops driving reliable improvement, and contrast it with the impractical precision of formal proof.
Why it matters: Repeated improvement needs feedback capable of recognizing gains the agent’s own judgment might miss.
Key terms: signal, precision, availability, gap, ground-truth signal, environment signal, labelled dataset, judge-based signal, absolute scoring, pairwise preference
1.4.2 The search family
This section compares five families of search methods by what they search, how they explore, and the effort they require, then explains how to choose and apply them across artifacts.
- Candidate search and exploration: Explain how search generates artifact variants and uses a signal to select among them, contrasting hill-climbing with pairwise and population methods.
Why it matters: The number of candidates a search maintains changes both its cost and what improvements it can find. - Reinforcement learning search fit: Describe how reinforcement-learning searches learn from outcomes and why they fit memory and tool-selection artifacts better than prompts or code.
Why it matters: The form of the artifact helps determine whether RL is a sensible search choice. - Evolutionary archive search: Explain what evolutionary archive search retains and why its broader source of candidates costs more compute.
Why it matters: An archive can revisit useful earlier variants instead of relying only on the current population. - Meta-search target: Distinguish meta-search from searches that seek a better artifact.
Why it matters: Meta-search differs from the other families in what it tries to improve, not just in how much it spends. - Search effort and reach: Explain the cost-versus-improvement-reach tradeoff and state the section's rule for choosing search effort.
Why it matters: Choosing a search method means weighing the improvement sought against the compute it will consume. - Search portability across artifacts: Explain why a named search technique need not be confined to the artifact layer used in its original demonstration, and identify the qualification for RL searches.
Why it matters: A published example should not unnecessarily limit which artifacts you consider improving with a search method.
Key terms: search, signal, candidate variants, hill-climbing, local peak, pairwise and population methods, reinforcement-learning searches, policy, episodic memory, tool selection
1.5 Summary
The chapter summary contrasts manual agent improvement with a closed self-improvement loop and recaps the harness artifacts, signals, and searches that make the loop possible.
Where this is used later
- The signal step of self-improvement (chapter 3) · The signal step of self-improvement is examined in detail.
- Implementing LLM-as-Judge for signal measurement (chapter 2) · Implements LLM-as-Judge for measuring a signal.
- Selecting the search method (chapter 2) · Introduces choices about how to search for artifact variants.
- Putting it together with Helix (chapter 2) · Puts measurement, search, and an artifact change together in an improvement workflow.
- The OfflineImprover (chapter 2) · Provides the offline-loop distinction used when introducing the OfflineImprover.
- Connecting the prompt artifact archive (chapter 2) · The prompt artifact archive depends on keeping track of artifact versions.
- Introducing the RAG agent (chapter 2) · Introduces what an improvement artifact is.
- Building the agent and text prompt artifact (chapter 2) · The prompt artifact example applies the L1 category.
Concepts explained in this chapter
- self-improving agentAn agent whose capabilities improve through changes to the shared instructions, tools, or other infrastructure around its model rather than changes to the model itself.
- searchSearch is an optimization process that generates candidate artifact variants and selects among them according to a signal.
- agent harnessSoftware scaffolding that surrounds an agent and provides its supporting execution environment.
- deployment gateA control point requiring review or approval before a proposed agent or harness change is deployed.
- code and tools layerThe deepest agent infrastructure layer, comprising the harness's source code, tool implementations, and orchestration logic.
- memory layerThe agent infrastructure layer that retains information learned across tasks, including entries, trajectories, values, and records of successful approaches.
- metacognitive layerThe third artifact layer, consisting of artifacts for an agent's monitoring or reasoning about its own processes.
- judge-based signalFeedback produced when a language model evaluates an agent’s output.
- layer-zero toolkitThe engineering machinery that runs an agent task loop and supports measuring gaps, searching for improvements, and managing variants.
- text/context layerThe shallowest editable infrastructure layer, consisting of text that instructs a model what to do on each call.
- availabilityThe ease and speed with which a signal can be obtained for a task, considering cost and required resources.
- environment signalAn environment signal evaluates an agent’s output using an outcome or evidence from the external world rather than a supplied label.
- ground-truth signalFeedback that evaluates an output by comparing it with a known correct answer.
- precisionThe accuracy with which a signal reports the size of the quality gap between an output and the desired result.
- process signalA feedback signal that evaluates the steps by which an agent reaches its output.
- offline loopAn agent-improvement cycle that evaluates proposed changes and applies them after a deployment review.
- archive searchAn evolutionary search method that breeds candidate variants from an archive of previously found good variants.
- hill climbingAn iterative search method that makes local changes to a candidate and retains changes that improve its score.
- meta-searchMeta-search is a search method that searches for a better search procedure rather than directly searching for a better artifact.
- pairwise and population searchSearch methods that compare candidates pairwise or select among a population of candidate variants.
- reinforcement-learning searchA search method that uses reinforcement learning to improve candidate artifacts or policies based on outcomes.
- measure-change-measure cycleA recurring process of measuring an agent's output, finding an improved version, and submitting it.
- LLM-as-a-judgeA pattern in which a large language model evaluates outputs against a rubric, potentially comparing candidates to select the best.
- promptText supplied to a language model as instructions or input for an interaction.
- reflection signalA feedback signal consisting of a verbal critique of an agent’s trajectory written by the agent or a critic.
- Darwin Gödel MachineDGM is an evolutionary search system that retains an archive of candidate code variants and uses it to generate and evaluate further variants.
- GEPAGEPA is a genetic-Pareto search method that breeds candidate variants reflectively from a population.
- SPOSPO is a pairwise search method that generates and compares candidate variants.
- rubricA set of criteria for evaluating an output.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
