A KNN optimizer selects prompt demonstrations separately for each language-model call. Instead of compiling one fixed set of examples for every input, it chooses the k training examples most similar to the current input. section 5.5 section 5.7
During optimization, the training examples are stored and encoded; no language-model calls are needed. The demonstrations are chosen later, when the program receives an input. Because the examples can be tailored to that input, fewer demonstrations may be needed, potentially saving time and money. section 5.5
Selecting examples for intent classification
For an utterance such as “What time is the flight today from Miami to Nashville?”, the optimizer selects similar training utterances as demonstrations. These can make the relevant intent clear without requiring examples for every intent the program might encounter. section 5.5
Figure 5.3 KNN optimizer algorithm. This algorithm first searches for two similar examples and then adds them to the prompt. Examples 1 and 2 have high similarity scores and are chosen as prompt demos, while examples 3 and 4 have low similarity and are left out of the prompt.
The diagram shows two high-similarity examples selected for the prompt while two less similar examples are left out. section 5.5
In DSPy, configure KNNFewShot with k, a training set, and a vectorizer. Compile it with the prediction module, then call the optimized module with an input. Inspecting the language model’s history reveals the prompt used for that call. section 5.5
Listing 5.12 KNNFewShot optimizer
import dspy
from dspy import KNNFewShot
from sentence_transformers import SentenceTransformer
encoder_func = SentenceTransformer(
"sentence-transformers/all-MiniLM-L12-v2",
token=False).encode
knn_optimizer = KNNFewShot(
k=3,
trainset=train_set,
vectorizer=dspy.Embedder(encoder_func)
)
optimized_module = knn_optimizer.compile(dspy.Predict(ClosedIntentSignature))
prediction = optimized_module(**dev_set[0].with_inputs())
print(lm.inspect_history(n=1))
This listing uses a SentenceTransformer encoder with k=3, compiles an intent-classification predictor, and inspects the resulting language-model call. section 5.5
