BootstrapFewShotWithRandomSearch automates trial-and-error in choosing prompt demonstrations: it tests multiple demo sets on validation data and selects the best-performing set. It works similarly to BootstrapFewShot, with additional controls for candidate search and evaluation. section 5.4
Configure the search and its evaluation:
- Set a metric and demo limits, including max_bootstrapped_demos of at least 1. section 5.4
- Use num_candidate_programs to control how many demo sets are evaluated. Supply valset for scoring; otherwise, DSPy splits the training set into training and validation subsets. section 5.4
- Set num_threads to limit parallel validation calls, and max_errors to control how many errors are allowed before compilation is canceled. Candidate prompts are evaluated in sequence; validation may run in parallel for each candidate. section 5.4
Listing 5.11 BootstrapFewShotWithRandomSearch
from dspy import BootstrapFewShotWithRandomSearch
optimizer = BootstrapFewShotWithRandomSearch(
metric=validate_answer,
max_bootstrapped_demos=10,
max_labeled_demos=5,
num_threads=10,
num_candidate_programs=10
)
intent_classifier_bootstrap_few_shot_with_random_search = optimizer.compile(
intent_classifier,
trainset=train_set,
valset=val_set)
pred = intent_classifier_bootstrap_few_shot_with_random_search(**dev_set[0].inputs())
print(lm.inspect_history(n=1))
The example configures ten candidate sets, up to ten bootstrapped demos, five labeled demos, and ten threads, then compiles an intent classifier against training and validation sets. section 5.4
