In DSPy bootstrapping, rejection sampling selects training examples whose predictions pass a metric, keeping them as demonstrations and rejecting failures. With an exact-match metric, only examples whose predicted labels match the supplied labels are accepted. section 5.3
The optimizer tests examples in training-set order until it finds enough usable demonstrations; additional rounds can test examples again. A metric can also score program quality, so its role in bootstrapping is distinct from its role in evaluation. section 5.3
Listing 5.6 BootstrapFewShot optimizer
from dspy import BootstrapFewShot
optimizer = BootstrapFewShot(
metric=validate_answer,
max_bootstrapped_demos=4,
max_labeled_demos=6,
max_rounds=10
)
prog_bootstrap_few_shot = optimizer.compile(student=intent_classifier,
trainset=train_set)
prediction = prog_bootstrap_few_shot(**dev_set[0].inputs())
print(lm.inspect_history(n=1))
Configure BootstrapFewShot with a metric and demonstration limits, then compile it against a training set. section 5.3
