liveBook · Manning

KNN optimizer

As explained in Building LLM Applications with DSPy

A KNN optimizer selects prompt demonstrations using nearest-neighbor similarity between an input and available examples.

A KNN optimizer selects prompt demonstrations separately for each language-model call. Instead of compiling one fixed set of examples for every input, it chooses the k training examples most similar to the current input. section 5.5 section 5.7

During optimization, the training examples are stored and encoded; no language-model calls are needed. The demonstrations are chosen later, when the program receives an input. Because the examples can be tailored to that input, fewer demonstrations may be needed, potentially saving time and money. section 5.5

Selecting examples for intent classification

For an utterance such as “What time is the flight today from Miami to Nashville?”, the optimizer selects similar training utterances as demonstrations. These can make the relevant intent clear without requiring examples for every intent the program might encounter. section 5.5

Figure 5.3 KNN optimizer algorithm. This algorithm first searches for two similar examples and then adds them to the prompt. Examples 1 and 2 have high similarity scores and are chosen as prompt demos, while examples 3 and 4 have low similarity and are left out of the prompt.
figure

The diagram shows two high-similarity examples selected for the prompt while two less similar examples are left out. section 5.5

In DSPy, configure KNNFewShot with k, a training set, and a vectorizer. Compile it with the prediction module, then call the optimized module with an input. Inspecting the language model’s history reveals the prompt used for that call. section 5.5

Listing 5.12 KNNFewShot optimizer
import dspy
from dspy import KNNFewShot
from sentence_transformers import SentenceTransformer

encoder_func = SentenceTransformer(
"sentence-transformers/all-MiniLM-L12-v2", 
token=False).encode

knn_optimizer = KNNFewShot(
    k=3,
    trainset=train_set,
    vectorizer=dspy.Embedder(encoder_func)
)

optimized_module = knn_optimizer.compile(dspy.Predict(ClosedIntentSignature))
prediction = optimized_module(**dev_set[0].with_inputs())
print(lm.inspect_history(n=1))

This listing uses a SentenceTransformer encoder with k=3, compiles an intent-classification predictor, and inspects the resulting language-model call. section 5.5

How Building LLM Applications with DSPy covers it

How deep it goes
Explains training-set encoding and per-call selection, then configures and compiles a KNNFewShot predictor.
Examples
Uses a flight-time utterance and similar training utterances to illustrate intent-classification demonstrations.
What you can do after
Apply it via DSPy KNNFewShot and inspect the prompt used for a prediction.
How it connects
Connects nearest-neighbor similarity with embeddings, few-shot prompting, and intent classification.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building LLM Applications with DSPy. The section links point to the parts of the book it draws on.