KNN optimizer

As explained in Building LLM Applications with DSPy

A KNN optimizer selects prompt demonstrations using nearest-neighbor similarity between an input and available examples.

A KNN (k-nearest neighbors) optimizer selects prompt demonstrations separately for each language-model call. Instead of compiling one fixed set of examples into every prompt, it finds the k training examples most similar to the current input. For an intent classifier receiving a question about a flight, demonstrations about similar flight questions are more useful than unrelated utterances. section 5.5

Compilation stores and encodes the training examples; it does not call the language model to choose demonstrations. Selection happens when the optimized program receives an input. Because each prompt needs examples relevant to that input rather than every possible input, it can often use fewer demonstrations, with modest time and cost savings. section 5.5

Figure 5.3 KNN optimizer algorithm. This algorithm first searches for two similar examples and then adds them to the prompt. Examples 1 and 2 have high similarity scores and are chosen as prompt demos, while examples 3 and 4 have low similarity and are left out of the prompt.

The diagram shows the central decision: high-similarity examples enter the prompt, while lower-similarity examples do not. section 5.5

Listing 5.12 KNNFewShot optimizer

import dspy
from dspy import KNNFewShot
from sentence_transformers import SentenceTransformer

encoder_func = SentenceTransformer(
"sentence-transformers/all-MiniLM-L12-v2", 
token=False).encode

knn_optimizer = KNNFewShot(
    k=3,
    trainset=train_set,
    vectorizer=dspy.Embedder(encoder_func)
)

optimized_module = knn_optimizer.compile(dspy.Predict(ClosedIntentSignature))
prediction = optimized_module(**dev_set[0].with_inputs())
print(lm.inspect_history(n=1))

The Python example creates a SentenceTransformer encoder, passes it to DSPy’s KNNFewShot with k=3 and a training set, then compiles and calls an intent predictor. section 5.5

Use KNN when the best demonstrations are likely to depend on the input. Other demonstration optimizers select examples randomly, bootstrap them, or search for a strong set; KNN instead changes the selected set from call to call. section 5.5 section 5.7

How Building LLM Applications with DSPy covers it

How deep it goes

Explains selection and compilation timing, then shows a DSPy KNNFewShot configuration; does not detail the similarity calculation.

Examples

Uses a flight-question intent example and a Python listing that compiles and calls a three-neighbor predictor.

What it assumes

Assumes a training set and a DSPy predictor, while explaining why nearest examples are chosen.

What you can do after

Apply it via DSPy KNNFewShot to select input-specific demonstrations from a training set.

How it connects

Connects nearest neighbors and text encoding to few-shot demonstrations, intent classification, and DSPy prompt optimization.

liveBookmanning.com

Read Building LLM Applications with DSPy on liveBook Get the book at manning.com

Related concepts

• k-nearest neighborsuses · K-nearest neighbors is a method that predicts a query’s class or value from the labels or values of its k closest training examples, as measured by a specified distance or similarity.
• training datasetuses · A collection of examples or observations supplied to a learning procedure to fit or adapt a model or learn its patterns or parameters.
• demonstrationuses · A demonstration is an example pairing an input, request, or prompt with a corresponding output, answer, or label to illustrate a task or expected behavior.
• LabeledFewShotcontrasts with · LabeledFewShot is a DSPy optimizer that constructs prompts by selecting labeled training examples without verifying language-model predictions.
• tokenproduces · A token is a discrete unit of text, such as a word, word fragment, character, or symbol, represented and processed as an element by a language model.

This explainer was generated by Manning's knowledge graph from the text of Building LLM Applications with DSPy. The section links point to the parts of the book it draws on.