Choosing prompt demonstrations by hand is difficult, especially when a program makes several language-model calls but its training examples contain only the initial inputs and final answers. Bootstrapping lets DSPy run the program on training examples, keep runs that pass a quality metric, and turn their language-model inputs and outputs into demonstrations. This is not bootstrap sampling: its purpose is to generate demonstration fields that were absent from the training data. section 5.3 section 5.3.1 section 5.7
With `BootstrapFewShot`, the main controls are:
- `metric` decides which completed runs qualify. The training examples must contain the fields that the metric checks; they need not contain fields generated during a run. section 5.3.1
- `max_bootstrapped_demos` limits demonstrations built from successful runs. `max_labeled_demos` controls demonstrations taken from labeled examples; setting it to zero isolates bootstrapping. section 5.3 section 5.3.1
- `max_rounds` bounds optimization attempts. Compile the optimizer with a student program and training set, then call the compiled program as usual. section 5.3
A missing reasoning field
An intent-classification example may supply a message and its correct `intent_label`, but no `reasoning`. With `ChainOfThought`, a successful language-model run supplies both outputs. If the metric accepts the label, DSPy can include the generated reasoning in the prompt demonstration. This also exposes a trade-off: a metric that checks only the label does not independently verify the reasoning. section 5.3.1
Listing 7.22 Optimizing a multihop module using BootstrapFewShot
from dspy import BootstrapFewShot
class PredictWithClearQuestion(dspy.Module): #1
def __init__(self, signature):
self.query1 = dspy.Predict("question -> clear_question")
self.query2 = dspy.Predict(signature)
def forward(self, **args):
q = args['message']
args['message'] = self.query1(question=q).clear_question
return self.query2(**args)
lm = dspy.LM(model='gpt-4o-mini')
dspy.configure(lm=lm)
prog = PredictWithClearQuestion(ClosedIntentSignature) #2
optimizer = BootstrapFewShot( #3
metric=validate_answer,
max_bootstrapped_demos=3,
max_labeled_demos=0,
max_rounds=1
)
prog_bootstrap_few_shot = optimizer.compile(student=prog,
trainset=train_set) #4
prediction = prog_bootstrap_few_shot(**dev_set[0].inputs()) #5
print(lm.inspect_history(n=2)) #6
The two-step Python example clarifies a question before classifying it. A successful complete run can supply a demonstration for each step, including the generated intermediate `clear_question` that connects them. section 7.6.1
For a multistep program, acceptance is based on the final result, while demonstrations retain inputs and outputs from the intervening calls. This makes it possible to optimize intermediate prompts without hand-labeling every intermediate value. It also means the metric’s coverage matters: a good final score causes DSPy to treat the intervening outputs as suitable demonstrations. section 7.6.1
