liveBook · Manning

GEPA

As explained in Building LLM Applications with DSPy

GEPA is a DSPy optimizer that reflectively generates prompt candidates using task information, prior prompts, and evaluation feedback.

GEPA is a DSPy optimizer for improving prompt instructions without rewriting each candidate by hand. It combines a genetic algorithm, a Pareto front, and reflection: generate candidate prompts, score them, and use promising candidates to produce the next round. section 6.5 section 6.9

The optimization loop has three practical parts:

  • Score each candidate on individual validation examples. GEPA prefers numeric metric scores; Boolean scores limit how well it can work. section 6.5.2
  • Keep prompts that score highest on at least one example in the Pareto front. Select from this pool for later mutations or merges, favoring prompts that lead on more examples. section 6.5.2
  • Use reflective mutation or system-aware merge to propose new prompts. GEPA considers the task, data, LM-call history, source code, available metric feedback, and saved candidate rationales rather than making arbitrary edits. section 6.5.3
Table 6.4 Maintaining the Pareto front after three steps
Prompt
Parent
Ex 1
Ex 2
Ex 3
Ex 4
Ex 5
In Pareto
1
–
0.5
0.6
0.7
0.4
0.1
False
2
1
0.6
0.7
0.8
0.5
0.0
True
3
2
0.4
0.3
0.2
0.3
0.2
True

The per-example scores show why one overall winner is not enough: the second prompt leads on four examples, while the third leads on the fifth. Both remain eligible; the first leaves the front. section 6.5.2

Optimize an airline-intent classifier

Start with a DSPy classifier, a training set, a validation set, and a metric that returns a numeric score with explanatory feedback. GEPA uses training examples for reflective prompt updates and validation examples to track scores and maintain the Pareto front; without a validation set, it uses the training set for both. Supply a reflection LM and set exactly one budget option: max_metric_calls, max_full_evals, or auto. A deliberately quick run took about one minute on Google Colab, used about 42,000 tokens, and made 108 LM calls. Its simplified signature also helped avoid generated intent labels outside the intended list. section 6.5.4

Reflection makes prompt generation more informed, but GEPA still requires repeated evaluations. Compare it with other optimizers on the task and LM at hand rather than assuming it will always win. section 6.5.3 section 6.0

How Building LLM Applications with DSPy covers it

How deep it goes
Explains candidate generation, per-example Pareto selection, and classifier configuration, without detailing the optimizer’s source-code internals.
Examples
Uses per-example prompt-score tables and an airline-intent classifier with numeric scoring and feedback.
What it assumes
Assumes an existing DSPy classifier and datasets, while explaining Pareto-front selection from an initial prompt.
What you can do after
Apply it via DSPy to optimize a classifier’s instructions using datasets, a feedback metric, a reflection LM, and an evaluation budget.
How it connects
Connects genetic algorithms and Pareto fronts with reflection, metric feedback, and the SIMBA optimizer.
Cover of Building LLM Applications with DSPy
liveBookmanning.com
Read Building LLM Applications with DSPy on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building LLM Applications with DSPy. The section links point to the parts of the book it draws on.