liveBook · Manning

LabeledFewShot

As explained in Building LLM Applications with DSPy

LabeledFewShot is a DSPy optimizer that selects labeled training examples as prompt demonstrations.

Choosing prompt examples by hand is difficult. LabeledFewShot offers a simple starting point: it randomly selects labeled examples from a training set and adds them as demonstrations to a DSPy program’s prompt. It does not search for the best set or test the selected examples against a validation set during compilation. section 5.2 section 5.2.1

Listing 5.3 Using the LabeledFewShot optimizer
from dspy.teleprompt import LabeledFewShot

num_examples = 4
labeled_few_shot_optimizer = LabeledFewShot(k=num_examples)
prog_labeled_few_shot = labeled_few_shot_optimizer.compile(student=intent_classifier, 
                                                           trainset=train_set)
prediction = prog_labeled_few_shot(**dev_set[0].inputs())
print(prediction)
print(lm.inspect_history(n=1))
print(prog_labeled_few_shot.demos)

Set k to the number of demonstrations, then compile the existing program with a training set. The result is a separate program that can be called like the original; inspect_history() shows its prompt, and .demos shows the selected examples. section 5.2.1

For intent classification, each demonstration pairs an input message with its intent label. Keep training examples available for selection and use separate validation or development examples to judge whether the resulting program improves accuracy. Because selection is random, a larger k does not guarantee a better prompt; more demonstrations also make prompts longer. section 5.2 section 5.2.1 section 5.6.1

Compare against the unoptimized program

With gpt-4o-mini and ChainOfThought, an unoptimized intent classifier scored 89.16% on 1,000 development examples. LabeledFewShot scored 92.35% with 10 prompt examples and 94.18% with 50. These results support trying several k values, not assuming that one random selection will be strongest. section 5.6.1

Use LabeledFewShot when a quick, straightforward demonstration-based baseline is useful. If selecting stronger examples matters, BootstrapFewShotWithRandomSearch tries candidate sets and selects among them; KNN instead selects demonstrations according to each input’s similarity. section 5.2 section 5.7

How Building LLM Applications with DSPy covers it

How deep it goes
Shows k, compile(), prompt inspection, and evaluation; does not present LabeledFewShot as a validation-guided search.
Examples
Uses an intent classifier, a four-demonstration compilation listing, and development-set accuracy comparisons across demonstration counts.
What it assumes
Assumes an existing DSPy program and labeled training examples; supplies the classifier setup and dataset preparation used for the example.
What you can do after
Apply it via DSPy to compile a program with labeled demonstrations, inspect its prompt, and compare its accuracy with an unoptimized program.
How it connects
Connects prompt demonstrations to training sets, validation sets, intent classification, ChainOfThought, and alternative demonstration optimizers.
Cover of Building LLM Applications with DSPy
liveBookmanning.com
Read Building LLM Applications with DSPy on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building LLM Applications with DSPy. The section links point to the parts of the book it draws on.