Demonstration optimization selects input-output examples to include in a language-model prompt. Given labeled examples, an optimizer chooses a subset to serve as demonstrations. This is distinct from optimizing the prompt’s instructions: the goal is to find examples that help the model recognize the task’s patterns and difficult cases. section 5.0 section 5.1
When choosing demonstrations for an intent classifier:
- Include labeled messages that distinguish classes such as Flight, Booking, Time, and Date. section 5.1
- Check class coverage. Examples drawn only from Flight may suggest that almost every message belongs to that class. section 5.1
- Test combinations rather than assuming an obvious example is useless. Basic examples may help alongside unusual or challenging ones. section 5.1
Figure 5.2 Prompt optimization. The input to the optimizer is a set of examples with their labels. During optimization, the optimizer selects a subset of the examples to use as prompt demonstrations.
Shows the optimization input—labeled examples—and the selected subset used as prompt demonstrations. section 5.1
Demonstrations often make prompts more effective, but also slightly longer. A practical way to judge a selected set is to compare predicted labels before and after optimization using DSPy’s evaluation framework. Instruction optimization remains a separate option, particularly when a subtle task needs detailed instructions. section 5.0 section 5.1
