Chapter 6: Optimizing prompt instructions

Introduction

The chapter introduces instruction optimization in DSPy, explains its meta-prompting foundation, and frames how to compare optimizers and their costs.

• Meta-prompting for instruction optimization: Explain how meta-prompting supports DSPy’s instruction optimization: one LM generates candidate prompts for another, and DSPy evaluates the candidates to find a strong performer.
  Why it matters: It connects instruction optimization to the demonstration optimization covered earlier.
• Optimizer approaches and progression: Describe the relationships among COPRO, MIPROv2, SIMBA, and GEPA at the level introduced here.
  Why it matters: These distinctions provide a starting point for understanding why the chapter examines several optimizers.
• Task-dependent optimizer choice and cost: Justify trying multiple optimizers for a task while accounting for the LM-call costs of doing so.
  Why it matters: A promising optimizer still needs to be tested on your task within a practical call budget.

Key terms: instruction optimization, demonstrations, meta-prompting, candidate prompts, OPRO, COPRO, MIPROv2, BootstrapFewShot, SIMBA, GEPA

6.1 COPRO

This section explains how COPRO improves prompt instructions by generating alternatives, evaluating them, and keeping the best candidate for the next round.

• COPRO instruction optimization cycle: Trace how COPRO starts with a program’s instructions, uses an LM to generate alternatives, evaluates them on a validation set, and makes the best-performing instructions the starting point for further variations.
  Why it matters: This cycle explains what COPRO is changing and how it chooses which instructions to keep.

Key terms: COPRO, prompt instructions, candidate prompt, validation set, coordinate ascent, hill climbing

6.1.1 What are the prefixes for the output fields?

This section explains how DSPy adapters construct prompts and what output-field prefixes are, including their limited role in COPRO optimization.

• Adapter-controlled prompt construction: Explain that DSPy adapters turn signatures, demonstrations, and other program information into prompt text, controlling its format and wording. Distinguish the mostly fixed prompt text from the parts optimizers can change.
  Why it matters: It clarifies which prompt text DSPy generates and which parts optimization can change.
• Output-field prefix role: Describe an output-field prefix as optional, brief guidance that some adapters place before an output field. Explain how its default and COPRO-suggested forms differ, and why instructions are generally more significant.
  Why it matters: It helps you recognize what COPRO may optimize without mistaking a prefix for a required or major part of every prompt.

Key terms: DSPy adapter, prompt template, signatures, demonstrations, prefix to the output fields, output fields, COPRO, instructions

6.1.2 What are the prompt instructions?

This section explains where DSPy places prompt instructions and why a signature docstring is useful even when an optimizer may later replace those instructions.

• Prompt instruction placement: Describe where prompt instructions appear in a DSPy prompt and explain why their placement matters.
  Why it matters: Knowing where instructions appear helps you interpret the prompt the LM receives.
• Signature docstring instructions: Explain how an unoptimized DSPy program gets its instructions and why providing a signature docstring remains useful when optimizing.
  Why it matters: A useful docstring gives the program better initial instructions and gives most optimizers a better place to start.

Key terms: prompt instructions, System message, demonstrations, signature, docstring, default instructions, optimizers

6.1.3 Executing the COPRO optimizer

This section shows how to run COPRO on a DSPy program, configure its prompt search and evaluation, and inspect the resulting scores.

• COPRO compilation workflow: Describe how to run COPRO and check its result: create the optimizer, call compile() on a DSPy program, then query the optimized program and inspect its history for updated instructions.
  Why it matters: The workflow connects COPRO’s optimization step to an observable change in the program.
• Candidate search controls: Explain how the metric, breadth, and depth govern COPRO’s search, including that the tested prompts include the original instructions.
  Why it matters: These settings determine both how COPRO compares prompts and how much search it performs.
• Evaluation dataset and settings: Explain what COPRO evaluates through compile()’s trainset and how eval_kwargs configures that evaluation.
  Why it matters: It clarifies what the supplied examples do in this optimization and where evaluation options belong.
• Candidate generation temperature: Describe what init_temperature controls and why the section suggests choosing it carefully or comparing runs at different values.
  Why it matters: Its value can affect the candidates COPRO tries, but COPRO does not tune it during a run.
• Optimization score tracking: Describe what track_stats exposes and how the resulting scores can help decide whether to continue optimization.
  Why it matters: Score trends give a way to judge whether more optimization may be worthwhile.

Key terms: COPRO, compile(), optimized program, history, metric function, breadth, depth, candidate prompts, trainset, validation set

6.1.4 Using a prompt LM

Explains how COPRO can use a separate LM to propose prompt instructions, how those prompts are evaluated, and how COPRO handles tied scores.

• Separate prompt and task LMs: Explain the distinct roles of the prompt LM and task LM in COPRO, and why using a powerful prompt LM with a cheaper production LM can be advantageous.
  Why it matters: Separating the LMs lets you improve a production prompt without making every production call use the more expensive model.
• Candidate evaluation call pattern: Trace why COPRO makes many task-LM calls alongside relatively few prompt-LM calls, and describe how the section says to inspect those calls.
  Why it matters: The call pattern explains both optimization cost and what you should expect to see when inspecting a run.
• Original prompt tie preference: Explain what COPRO does when a candidate and the original prompt receive identical scores, and why ties may occur with a small validation set.
  Why it matters: A tie does not necessarily mean an optimized candidate will replace the original prompt.

Key terms: prompt LM, task LM, compilation, candidate instructions, validation set, iteration, inspect_history(), dspy.inspect_history(n=1000), original prompt, identical score

6.1.5 How COPRO generates candidate instructions

This section explains how COPRO asks a prompt LM for candidate instructions and output prefixes, first from the original prompt and then from scored attempts.

• First-iteration candidate generation: Describe how COPRO generates candidates in its first iteration: it gives the prompt LM the original instructions and asks for improved instructions and an output-field prefix.
  Why it matters: It establishes what COPRO gives the prompt LM before it has scored earlier candidates.
• Multiple responses per request: Explain why COPRO uses Predict’s n parameter and why the first iteration requests breadth-1 generated responses.
  Why it matters: It explains both the efficient generation of several variations and the first iteration’s candidate count.
• Scored-history candidate generation: Trace what COPRO supplies to the prompt LM from the second iteration onward and what it requests in return.
  Why it matters: It shows how later requests incorporate the results of earlier candidate tests.

Key terms: candidate instructions, original instructions, prompt LM, prefix for the output fields, Predict, n, breadth-1, original prompt, attempted instructions, instructions

6.1.6 Executing multiple optimizers

This section explains how to chain optimizers so a DSPy program can gain demonstrations and then have its instructions updated.

• Chaining complementary prompt optimizers: Explain why COPRO can be paired with a demonstration optimizer and describe the example sequence: LabeledFewShot adds demonstrations to the original program, then COPRO updates its instructions. State that an optimizer can compile an already optimized program, though further improvement is not guaranteed.
  Why it matters: Chaining lets you address both demonstrations and instructions instead of expecting COPRO to optimize both.

Key terms: COPRO, LabeledFewShot, demonstrations, instructions, compile

6.2 MIPROv2

This section explains how MIPROv2 jointly searches for demonstrations and instructions, configures that search, and evaluates candidate prompts to select a final program.

• Joint prompt component optimization: Explain why MIPROv2 optimizes demonstrations and instructions together rather than tuning them one after the other.
  Why it matters: It explains why MIPROv2 searches over whole prompts instead of treating examples and instructions as independent choices.
• Three-stage candidate search: Trace how MIPROv2 creates demonstration and instruction candidates and then combines them.
  Why it matters: The three stages show what MIPROv2 is searching over before it chooses a prompt.
• Search budget configuration: Describe how the required metric and optional search settings control MIPROv2’s configuration, cost, and likely results.
  Why it matters: These settings let you balance search effort against the importance of improving the prompt.
• Validation trials and selection: Describe how MIPROv2 evaluates candidate combinations during trials and chooses its final prompt.
  Why it matters: It distinguishes quick screening scores from the validation result used to select the final prompt.
• Prompt and task LM roles: Distinguish the prompt LM’s role from the task LM’s role when separate models are configured.
  Why it matters: It clarifies which model generates instruction ideas and which model incurs most evaluation calls.

Key terms: MIPROv2, demonstrations, instructions, BootstrapFewShot, COPRO, zero-shot, candidate instructions, prompt LM, task LM, Bayesian optimization

6.2.1 Grounding

This section explains how MIPROv2 grounds instruction generation in task information, how its grounding options work, and why it cannot use evaluation history at that stage.

• Grounded instruction generation: Explain how MIPROv2 prepares the prompt LM to generate task-relevant instructions by supplying information about the program and training data.
  Why it matters: Grounding helps candidate instructions reflect the actual task rather than a generic description of it.
• Purpose of grounding: Justify why MIPROv2 grounds candidate instructions before evaluating them.
  Why it matters: Better-informed candidates make costly instruction evaluations more worthwhile.
• Grounding controls and timing: Describe how tips and proposer settings affect MIPROv2’s grounding, and distinguish its instruction search from COPRO’s use of call history.
  Why it matters: These distinctions clarify what information can guide instruction proposals before any candidates have been evaluated.

Key terms: MIPROv2, grounding, prompt LM, task LM, dataset summary, source code, candidate instructions, evaluation, tips, tip_aware_proposer

6.2.2 Using Bayesian hyperparameter tuning for trial optimization

This section explains how MIPROv2 uses Bayesian optimization to choose promising combinations of demonstrations and instructions without evaluating every possible prompt.

• Adaptive prompt combination search: Explain why MIPROv2 does not test every combination of demonstration sets and instructions, and how results from earlier trials guide what it tests next.
  Why it matters: Adaptive selection makes searching many candidate prompts practical without paying to evaluate them all.
• Exploration and exploitation balance: Distinguish the combinations MIPROv2 favors during early exploration from those it favors later during exploitation.
  Why it matters: This balance explains why an optimizer sometimes tests an uncertain prompt instead of the current favorite.
• Optuna surrogate feedback cycle: Describe how Optuna’s surrogate estimates help select a trial and how the actual evaluation improves subsequent selections.
  Why it matters: The surrogate lets MIPROv2 compare possible next trials cheaply while still checking selected prompts against the real task LM.

Key terms: MIPROv2, demonstrations, instructions, Bayesian optimization, trials, exploration, exploitation, uncertainty, Optuna, surrogate function

6.3 InferRules

This section explains how InferRules turns patterns in training examples into prompt rules, why those rules may be useful, and how its example run is configured.

• Rules in prompt instructions: Explain that InferRules adds demonstrations and generates rules connecting input patterns to output fields, then puts those rules into the task prompt’s instructions. Distinguish this from MIPROv2’s use of grounding information.
  Why it matters: It identifies what InferRules changes in the prompt and how that differs from MIPROv2.
• Rule efficiency and reliability: Justify why including rules can use fewer input tokens than many full demonstrations, and state the condition on that advantage.
  Why it matters: The potential token savings are useful only when the generated guidance is dependable.
• InferRules run configuration: Describe the example’s rule-candidate settings and how InferRules obtains training and validation sets when only a train set is supplied.
  Why it matters: These settings determine the rule candidates considered and clarify what data the example uses.

Key terms: InferRules, demonstrations, rules, instructions, grounding, prompt LM, input tokens, num_candidates, num_rules, train

6.4 SIMBA

Introduces SIMBA as a reflective prompt optimizer that iteratively adds demonstrations or rules, while noting that optimizer performance depends on the task and language model.

• SIMBA’s reflective optimization process: Describe how SIMBA improves a prompt over a series of iterations by analyzing its performance and adding either one demonstration or one rule at a time.
  Why it matters: This is the basic idea behind SIMBA before its application is explored.
• Task-dependent optimizer choice: Explain the qualification on calling SIMBA and GEPA strong optimizers: the best optimizer for a particular application depends heavily on the task and language model.
  Why it matters: A strong general reputation is not a guarantee that an optimizer is best for a specific application.

Key terms: SIMBA, demonstrations, rules, reflective process, GEPA, optimizer, LM

6.4.1 Optimizing the airline classifier with SIMBA

This section shows how SIMBA optimizes an airline classifier by testing candidate prompts, reflecting on their performance, and adding rules or demonstrations, while cautioning that optimization scores are not final evaluation results.

• SIMBA iteration and candidate selection: Trace how SIMBA selects a minibatch and candidate prompt, tests the prompt, and adds a revised candidate to its pool.
  Why it matters: This explains how SIMBA explores improvements rather than committing to one prompt immediately.
• Grounded reflection on trajectories: Explain what information SIMBA gives the prompt LM and how contrasting execution trajectories helps it suggest improvements.
  Why it matters: The feedback is tied to observed program behavior rather than generic prompt advice.
• Training scores and separate evaluation: Explain the dataset requirement for SIMBA's compile() method and why its optimization metrics do not replace separate evaluation.
  Why it matters: Separate evaluation helps prevent a promising optimization score from being mistaken for reliable performance.
• SIMBA configuration parameters: Describe what num_candidates, max_steps, and max_demos control in this SIMBA example.
  Why it matters: These settings determine how broadly and how long SIMBA searches, and how many examples a prompt can retain.
• Airline classifier rule outcome: Describe what kind of change SIMBA made to the final airline-classifier prompt and illustrate it with one of the section's rules.
  Why it matters: The resulting rules make SIMBA's prompt changes concrete for the airline-classification task.

Key terms: minibatch, candidate prompt, pool, demo, rule, grounded process, prompt LM, trajectories, suggested rule, compile()

6.4.2 Taking advantage of feedback

Explains how metric-function feedback helps SIMBA improve prompts, using an intent-classification example and a more complex metric to show why explanations are useful alongside scores.

• Explanatory metric feedback: Explain what feedback means in this section and why SIMBA can use it to improve prompts more readily than a score alone.
  Why it matters: An explanation gives the optimizer a clearer improvement signal than a score by itself.
• Unspecified label optimization example: Describe how the SIMBA example uses feedback and a character-overlap score when the signature omits the correct set of labels.
  Why it matters: The example shows how explanatory feedback can help even when a task omits guidance the optimizer would otherwise need.
• Feedback for multidimensional metrics: Explain why feedback matters when a metric combines several checks into one score.
  Why it matters: Explaining individual checks makes a composite score more useful for prompt optimization.

Key terms: feedback, metric function, reflective optimizers, SIMBA, signature, labels, character-level overlap, numeric scores, Boolean scores, metric functions

6.5 GEPA

The section introduces GEPA as a DSPy optimizer, identifies its named foundations, and notes how its developers recommend using it.

• GEPA optimizer overview: Describe GEPA’s place in DSPy, what its name stands for, and the choice between using it standalone or within DSPy. Identify its named foundations without explaining how they work.
  Why it matters: This overview gives you the right context for the concepts explained in the following sections.

Key terms: GEPA, Genetic-Pareto, genetic algorithm, Pareto front, reflection, DSPy

6.5.1 Genetic algorithms

This section explains how genetic algorithms optimize prompts by maintaining and refining a pool of candidates rather than improving just one prompt.

• Candidate pools versus hill climbing: Distinguish genetic algorithms from hill climbing in prompt optimization and explain why a pool of candidates can make the search more robust.
  Why it matters: The pool explains both how the search differs from hill climbing and why it may be more robust.
• Initial prompts and variation operations: Describe how the initial prompt pool is formed and distinguish mutation from crossover, including the stated difference in GEPA.
  Why it matters: These operations show how evaluated prompts give rise to new candidates.
• Candidate pool size tradeoff: Explain why the candidate pool generally favors top-performing prompts and why its size matters.
  Why it matters: Pool size affects whether the search retains useful alternatives without being crowded by weak candidates.
• Exploration and exploitation in selection: Explain how candidate selection shifts between exploration and exploitation over iterations.
  Why it matters: The selection strategy determines whether the search tries varied promising approaches or concentrates on its best results.

Key terms: genetic algorithms, hill climbing, pool of candidate solutions, initial pool of candidates, mutating, crossover, GEPA, pool, top-performing prompts, diversity

6.5.2 The Pareto front

This section explains how GEPA uses per-example validation scores to maintain a Pareto front and select prompts for further optimization.

• Numeric validation scores: Explain why GEPA uses numeric metric scores when evaluating prompts on validation examples.
  Why it matters: The per-example scores are the basis for deciding which prompts remain worth exploring.
• Pareto front membership: Describe how per-example scores determine which prompts stay on GEPA’s Pareto front and when a prompt is displaced.
  Why it matters: The front keeps prompts with distinct strengths instead of retaining only one overall favorite.
• Front-based prompt selection: Explain how GEPA selects prompts from the Pareto front for mutation or crossover, and why it does not always select only the strongest prompt.
  Why it matters: Selection gives successful prompts more chances without abandoning other promising directions.
• Iterative front updates: Trace how GEPA evaluates a new candidate and updates the Pareto front across iterations, including the effect of validation-set size.
  Why it matters: The evaluation sequence and validation-set size shape how many candidate prompts GEPA can retain.

Key terms: GEPA, metric function, numeric scores, validation set, Pareto front, dominates, mutation, crossover, exploration, exploitation

6.5.3 Generating new candidates

This section explains how GEPA uses reflection to generate prompt candidates through mutation and crossover, and why reflection helps maintain a diverse pool of prompts.

• Reflective prompt mutation: Explain how GEPA uses evidence from earlier work to mutate prompts rather than changing them randomly.
  Why it matters: It explains why GEPA's prompt changes are informed by the application rather than arbitrary edits.
• Rationale-guided prompt crossover: Describe how GEPA combines two prompts using the rationales saved for its candidates.
  Why it matters: The saved explanations give GEPA a basis for combining prompts rather than merely splicing their text.
• Reflection and prompt diversity: Explain the benefit the DSPy team found from using reflection alongside the Pareto front.
  Why it matters: It shows that reflection helps preserve a range of candidates for the search, not just revise individual prompts.

Key terms: GEPA, reflective mutation, LM calls, metric functions, crossover, system-aware merge, rationale, candidate prompt, reflection, Pareto front

6.5.4 Using GEPA for the airline classifier

Shows how to configure GEPA for an airline classifier, choose its data and optimization budget, and inspect the resulting instructions.

• Classifier metric and signature setup: Explain how the airline-classifier example prepares its metric and signature for GEPA.
  Why it matters: These choices make the classifier example suitable for GEPA’s scoring and feedback.
• Training and validation set roles: Distinguish what GEPA does with the training and validation sets, including what happens if no validation set is supplied.
  Why it matters: The two sets have different roles even though GEPA can fall back to one set for both.
• Reflection language model role: Explain what the reflection_lm parameter supplies and why the example uses a strong model.
  Why it matters: It clarifies which model is responsible for proposing revised prompts.
• GEPA optimization budget options: Describe how the example sets GEPA’s budget and what its auto settings control.
  Why it matters: The budget determines the extent of the search without changing the validation-set size.
• Generated instruction inspection: Describe how the example views GEPA’s generated instructions and what is notable about their contents.
  Why it matters: Knowing where the demonstrations appear helps a reader inspect what GEPA actually produced.

Key terms: pred_name, pred_trace, metric function, SimpleSignature, train set, validation set, reflective updates, Pareto front, reflection_lm, prompt LM

6.6 Ensemble

This section explains how DSPy combines multiple programs into an ensemble, reduces their outputs to one prediction, and selects a small set of programs that works well together.

• Ensemble execution and tradeoffs: Explain that DSPy’s Ensemble optimizer creates a program from two or more existing programs, and that the resulting program runs them at inference time to combine their results. Describe the potential gain in quality alongside the higher cost of running multiple programs.
  Why it matters: It clarifies why a quick compilation step can still produce a more costly application at inference time.
• Reduction of program outputs: Describe the reduce function’s role in turning multiple program outputs into one set of output fields. Contrast the section’s majority-vote classifier example with its custom rule that returns a label only on full agreement and otherwise returns "Uncertain".
  Why it matters: The reduction rule determines how the ensemble turns several answers into the answer the application returns.
• Complementary ensemble selection: Explain how per-example validation predictions can help select a small ensemble of reasonably accurate programs whose errors differ, rather than including every available program. Describe how the example’s three-program subset matches the five-program majority-vote score.
  Why it matters: A smaller, complementary set may preserve the ensemble’s accuracy without running every program you have compiled.

Key terms: Ensemble optimizer, inference time, programs, reduce function, majority, Prediction, Uncertain, validation set, majority vote, complement each other

6.7 Saving the programs generated by DSPy

This section explains why to save optimized DSPy programs and contrasts saving a full program with saving its metadata.

• Purpose of saving optimized programs: Explain why an optimized DSPy program is saved to disk during experimentation and for production.
  Why it matters: Saving lets you reuse the results of optimization rather than compile the program again.
• Full program and metadata saves: Distinguish saving the full optimized program from saving only its metadata, including what each produces and what is needed to re-create the program from metadata.
  Why it matters: The two options differ in whether you store the full program or inspectable information for re-creating it.

Key terms: optimized program, compilation, pickle file, metadata, save_program, JSON file, demos, instructions

6.8 Comparing optimizers

The section explains how to compare prompt optimizers for an application and interprets results from two language models without treating any optimizer as universally best.

• Application Focused Optimizer Comparison: Explain why the aim is to find a strong prompt for the application rather than crown a universally best optimizer, and how available time and required quality affect comparisons.
  Why it matters: This keeps optimization effort focused on the prompt quality your application needs.
• Parameter Sensitive Optimizer Results: Explain why an optimizer's results cannot be separated from its hyperparameters, and describe why published rankings are only a starting assumption.
  Why it matters: A disappointing run may reflect its settings or the particular task rather than the optimizer alone.
• Two Model Experiment Takeaways: Describe what the reported tests showed for the unoptimized programs, COPRO, and MIPROv2, and what the gpt-4.1-nano result suggests about prompt tuning.
  Why it matters: The results illustrate why testing specific model, module, and optimizer combinations is more useful than assuming a winner.

Key terms: optimizers, hyperparameters, prompt, optimization time, diminishing returns, breadth, depth, heavy, demos, GEPA

6.9 Summary