A demonstration is an input paired with an appropriate output inside a language-model prompt. It shows the model what a successful response looks like. One demonstration makes a prompt one-shot; two or more make it few-shot. An explanation of why the output is appropriate can also help. section 2.1.2
Customer intent classification
Input: “I want to end my subscription.” Appropriate output: “Cancel Subscription.” This pair illustrates the intended classification more concretely than a label name alone. section 2.1.2 section 5.1
In DSPy, an example is a record in a dataset; a demonstration is a record selected for inclusion in a prompt. An optimizer can select demonstrations from labeled training examples, generate them by bootstrapping successful program runs, or choose them according to similarity to the current input. This makes selection testable rather than a matter of manually guessing which pairs will help. section 5.1 section 5.7
Choose the selection method to fit the application:
- LabeledFewShot selects labeled training examples; BootstrapFewShot can instead create demonstrations from runs that meet a metric. section 5.7
- BootstrapFewShotWithRandomSearch tries multiple sets and keeps the best; KNN selects similar demonstrations separately for each model call. section 5.7
- For a program with multiple model calls, bootstrapping can retain intermediate inputs and outputs as demonstrations for the corresponding steps. section 7.6.1
A candidate training set can be larger than the few demonstrations ultimately placed in a prompt. Validate candidate sets rather than assuming more is better: many-shot prompts cost more tokens and can sometimes confuse the model. In an intent-classification comparison, optimized prompts improved on the unoptimized baselines; KNN remained competitive with 10–15 demonstrations, though it added inference-time work. section 4.1.3 section 5.6.1
Listing 7.22 Optimizing a multihop module using BootstrapFewShot
from dspy import BootstrapFewShot
class PredictWithClearQuestion(dspy.Module): #1
def __init__(self, signature):
self.query1 = dspy.Predict("question -> clear_question")
self.query2 = dspy.Predict(signature)
def forward(self, **args):
q = args['message']
args['message'] = self.query1(question=q).clear_question
return self.query2(**args)
lm = dspy.LM(model='gpt-4o-mini')
dspy.configure(lm=lm)
prog = PredictWithClearQuestion(ClosedIntentSignature) #2
optimizer = BootstrapFewShot( #3
metric=validate_answer,
max_bootstrapped_demos=3,
max_labeled_demos=0,
max_rounds=1
)
prog_bootstrap_few_shot = optimizer.compile(student=prog,
trainset=train_set) #4
prediction = prog_bootstrap_few_shot(**dev_set[0].inputs()) #5
print(lm.inspect_history(n=2)) #6
The Python listing compiles a two-step DSPy module with BootstrapFewShot, showing where the metric, demonstration limits, and training set enter the workflow. section 7.6.1
