A prompt optimizer that keeps only one candidate can stall even when its score says the answer is wrong: the score does not identify the missed rule. GEPA (Genetic-Pareto) addresses both limits by keeping a population of agent prompts and reading execution traces before proposing edits. It changes instructions, not model weights. section 4.2.1
GEPA repeats a vary–select–remember loop:
- Vary: reflect on an agent’s trace to target a failure, then rewrite a prompt; crossover can also combine two parents. section 4.2.1
- Select: run and score new prompts, retaining candidates on a Pareto front. section 4.2.1
- Remember: carry a population forward rather than one winner, while discarding candidates outside the surviving pool. section 4.2.1 section 4.4.3
Figure 4.4 Introducing GEPA. The vary-select-remember cycle from the last section runs again here: vary is a reflected rewrite of the prompt plus crossover, select runs and scores each new prompt and keeps the best on a Pareto front, and remember carries a whole population forward. The one new piece is reflection inside the vary step, which reads the agent's trace so the rewrite targets the exact rule it skipped, and none of it touches the model's weights.

The diagram separates trace-guided rewriting, scoring and population retention without implying any change to model weights. section 4.2.1
When a population still stalls
Consider prompts for a revenue-leaderboard agent tested against six hidden rules. Reflection gives each proposed rewrite evidence from a run, but it cannot guarantee the missing rule will be found. In one recorded comparison, GEPA’s breeding pool settled on two survivors; later children repeatedly inherited the same unresolved failure. A differently worded prompt was not necessarily a different solution. section 4.2.1 section 4.4.3
GEPA spends more than a single-candidate hill-climb to explore multiple prompts. Unlike a Darwin Gödel machine archive, its pool drops candidates, so the search can lose access to an earlier branch. A highest-scoring artifact is still not automatically the one deployed: promotion remains a deliberate human gate. section 4.0 section 4.4.3
Figure 4.16 Convergence above memory: both runs in the Generations view. Each lane is one generation, and the chips in its gutter are the breeding pool at that moment: teal was picked as a parent, plain was available, struck through was already discarded. GEPA's pool freezes at two survivors while its discard pile grows, so every new child reworks the same two parents; DGM's pool only ever widens, and its champion ends a column the sampler reached back to after the rival branch burned two dead ends, which is the chapter's argument made literal.

The generation view makes GEPA’s shrinking pool and discarded branches visible beside an archive search that retains earlier candidates. section 4.4.3

