liveBook · Manning

GEPA

As explained in Self-Improving Agents

GEPA is a genetic-Pareto search method that breeds candidate variants reflectively from a population.

A prompt optimizer that keeps only one candidate can stall even when its score says the answer is wrong: the score does not identify the missed rule. GEPA (Genetic-Pareto) addresses both limits by keeping a population of agent prompts and reading execution traces before proposing edits. It changes instructions, not model weights. section 4.2.1

GEPA repeats a vary–select–remember loop:

  • Vary: reflect on an agent’s trace to target a failure, then rewrite a prompt; crossover can also combine two parents. section 4.2.1
  • Select: run and score new prompts, retaining candidates on a Pareto front. section 4.2.1
  • Remember: carry a population forward rather than one winner, while discarding candidates outside the surviving pool. section 4.2.1 section 4.4.3
Figure 4.4 Introducing GEPA. The vary-select-remember cycle from the last section runs again here: vary is a reflected rewrite of the prompt plus crossover, select runs and scores each new prompt and keeps the best on a Pareto front, and remember carries a whole population forward. The one new piece is reflection inside the vary step, which reads the agent's trace so the rewrite targets the exact rule it skipped, and none of it touches the model's weights.

The diagram separates trace-guided rewriting, scoring and population retention without implying any change to model weights. section 4.2.1

When a population still stalls

Consider prompts for a revenue-leaderboard agent tested against six hidden rules. Reflection gives each proposed rewrite evidence from a run, but it cannot guarantee the missing rule will be found. In one recorded comparison, GEPA’s breeding pool settled on two survivors; later children repeatedly inherited the same unresolved failure. A differently worded prompt was not necessarily a different solution. section 4.2.1 section 4.4.3

GEPA spends more than a single-candidate hill-climb to explore multiple prompts. Unlike a Darwin Gödel machine archive, its pool drops candidates, so the search can lose access to an earlier branch. A highest-scoring artifact is still not automatically the one deployed: promotion remains a deliberate human gate. section 4.0 section 4.4.3

Figure 4.16 Convergence above memory: both runs in the Generations view. Each lane is one generation, and the chips in its gutter are the breeding pool at that moment: teal was picked as a parent, plain was available, struck through was already discarded. GEPA's pool freezes at two survivors while its discard pile grows, so every new child reworks the same two parents; DGM's pool only ever widens, and its champion ends a column the sampler reached back to after the rival branch burned two dead ends, which is the chapter's argument made literal.

The generation view makes GEPA’s shrinking pool and discarded branches visible beside an archive search that retains earlier candidates. section 4.4.3

How Self-Improving Agents covers it

How deep it goes
Explains reflection, crossover, scoring and population retention through a search loop and recorded lineage, without detailing a full implementation.
Examples
Uses a revenue-leaderboard agent with six hidden rules and a lineage where two surviving prompts repeatedly produce the same failure.
What it assumes
Builds on single-candidate prompt search while introducing evolutionary search from its basic vary–select–remember loop.
What you can do after
Evaluate/understand it by tracing how prompts are proposed and retained, and diagnosing convergence from discarded branches and repeated checker feedback.
How it connects
Connects GEPA to evolutionary search, hill-climbing, Pareto fronts, crossover and Darwin Gödel machine archive search.

Also explained in

Compare how 2 books explain GEPA →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Improving Agents. The section links point to the parts of the book it draws on.