What you will learn in Self-Improving Agents
How to engineer adaptive harnesses
Chapter by chapter
Chapter 1: Foundations of self-improving agents
The introduction explains why production agents need ongoing improvement and presents a self-run improvement loop that does not retrain the underlying model.
What you learn in chapter 1 →Chapter 2: Getting started improving agents
The chapter introduces an offline improvement loop for a RAG agent’s prompt instruction and previews the tools used to build it.
What you learn in chapter 2 →Chapter 3: The Gödel Machine, and the importance of signal
The introduction contrasts the Gödel machine’s proof-based self-improvement with the practical approach of measuring changes and searching through candidates.
What you learn in chapter 3 →Chapter 4: Darwin Gödel Machine, improving search
The introduction frames evolutionary search as a way to improve candidate generation beyond a simple hill-climber, while noting its added cost.
What you learn in chapter 4 →Concepts explained in this book
- searchSearch is an optimization process that generates candidate artifact variants and selects among them according to a signal.
- evolutionary computationA search method that improves candidate solutions through variation and selection across iterations.
- Darwin Gödel MachineDGM is an evolutionary search system that retains an archive of candidate code variants and uses it to generate and evaluate further variants.
- self-improving agentAn agent whose capabilities improve through changes to the shared instructions, tools, or other infrastructure around its model rather than changes to the model itself.
- hill climbingAn iterative search method that makes local changes to a candidate and retains changes that improve its score.
- LLM-as-a-judgeA pattern in which a large language model evaluates outputs against a rubric, potentially comparing candidates to select the best.
- reflection signalA feedback signal consisting of a verbal critique of an agent’s trajectory written by the agent or a critic.
- populationA population is a collection of candidate solutions maintained together during a search process.
- ground-truth signalFeedback that evaluates an output by comparing it with a known correct answer.
- Pareto frontierA Pareto front is the set of non-dominated candidates in a multi-objective search, where improving one objective requires worsening another.
- archiveAn archive is a stored collection of past candidates retained during a search process.
- crossoverCrossover is an evolutionary operation that combines material from two parent candidates to produce a child candidate.
- archive searchAn evolutionary search method that breeds candidate variants from an archive of previously found good variants.
- smooth scorerA scoring function that awards graded partial credit according to how close an answer is to the desired result.
- GEPAGEPA is a genetic-Pareto search method that breeds candidate variants reflectively from a population.
- agent harnessSoftware scaffolding that surrounds an agent and provides its supporting execution environment.
- SPOSPO is a pairwise search method that generates and compares candidate variants.
- quality-diversity samplingA sampling strategy that favors high-quality archive entries while accounting for how often each entry has been reused.
- dominanceA relation in multi-objective optimization in which one candidate is at least as good on every objective and strictly better on at least one.
- mutation operatorMutation is an operation that makes a small change to a candidate solution in an evolutionary search.
- proof gateA binary verifier that compares an agent’s claim with a computable truth and classifies it as proven or unproven.
- availabilityThe ease and speed with which a signal can be obtained for a task, considering cost and required resources.
- precisionThe accuracy with which a signal reports the size of the quality gap between an output and the desired result.
- swap-and-agreeA pairwise judging technique that evaluates answers in both possible orders and accepts a verdict only when the judgments agree.
- code and tools layerThe deepest agent infrastructure layer, comprising the harness's source code, tool implementations, and orchestration logic.
- memory layerThe agent infrastructure layer that retains information learned across tasks, including entries, trajectories, values, and records of successful approaches.
- metacognitive layerThe third artifact layer, consisting of artifacts for an agent's monitoring or reasoning about its own processes.
- offline loopAn agent-improvement cycle that evaluates proposed changes and applies them after a deployment review.
- judge-based signalFeedback produced when a language model evaluates an agent’s output.
- local optimumA local peak is a point in a search landscape that scores higher than nearby alternatives but may be lower than points elsewhere.
- promptText supplied to a language model as instructions or input for an interaction.
- composite signalA feedback signal that combines multiple kinds of evaluation information, such as a score and a critique.
- GapMeasurementA structured evaluation result that records an assessment of the gap between a candidate output and a desired standard, along with supporting feedback.
- layer-zero toolkitThe engineering machinery that runs an agent task loop and supports measuring gaps, searching for improvements, and managing variants.
- text/context layerThe shallowest editable infrastructure layer, consisting of text that instructs a model what to do on each call.
- measure-change-measure cycleA recurring process of measuring an agent's output, finding an improved version, and submitting it.
- rubricA set of criteria for evaluating an output.
- deployment gateA control point requiring review or approval before a proposed agent or harness change is deployed.
- selectionAn evolutionary search operator that evaluates candidates and chooses among them according to their fitness.
- environment signalAn environment signal evaluates an agent’s output using an outcome or evidence from the external world rather than a supplied label.
- process signalA feedback signal that evaluates the steps by which an agent reaches its output.
- lineage treeA lineage tree is a tree-shaped record of candidate variants and their parent-child ancestry during a search.
- variationThe creation of a changed candidate from an existing candidate in an evolutionary search.
- signal ceilingThe limitation that any individual evaluative signal captures only part of the gap between an agent’s output and the desired result.
- meta-searchMeta-search is a search method that searches for a better search procedure rather than directly searching for a better artifact.
- pairwise and population searchSearch methods that compare candidates pairwise or select among a population of candidate variants.
- reinforcement-learning searchA search method that uses reinforcement learning to improve candidate artifacts or policies based on outcomes.
- search landscapeA search landscape is a space of candidate solutions characterized by their objective values and the routes available between them.
- position biasA systematic influence of an answer's position in a comparison on a judge's evaluation, independent of answer quality.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
