When labeled examples are scarce, few-shot learning tests whether a model can perform a new task after seeing only a small number of worked examples. In N-shot prompting, those examples appear directly in the prompt. Zero-shot prompting supplies an instruction but no worked examples. section 6.0
Two-shot translation
A zero-shot prompt might say, “Translate to French: hello.” A two-shot prompt first gives “English: hello → French: bonjour” and “English: dog → French: chien,” then asks for another translation. The examples demonstrate both the task and the expected answer format. section 6.0
In retrieval-augmented generation, “few-shot” can also describe a different setup: fine-tuning a retriever–reader system on a small labeled task set. These examples update the system rather than merely appearing in each prompt. Keeping the two setups distinct matters when comparing results or choosing an implementation. section 6.0 section 6.3.1
For few-shot fine-tuning of a RAG system, limited labels make the starting architecture and training signal consequential:
- Use a reader that can process multiple documents, such as Fusion-in-Decoder, and favor retriever–reader components jointly pre-trained to cooperate. section 6.5
- Train retrieval toward documents useful to the reader; Perplexity Distillation provides one such signal. section 6.5
- With fewer than 1,000 examples, consider updating only the query encoder while freezing the document encoder to limit overfitting. section 6.5
Retrieval can reduce reliance on knowledge stored in model parameters, but it cannot recover content absent from the retrieved documents. In a reported 64-example fine-tuning comparison on Natural Questions, an 11B-parameter Atlas model scored 42.4% Exact Match against 39.6% for a 540B-parameter PaLM model. That result supports the value of retrieval in this setting; it does not make prompt examples and fine-tuning interchangeable. section 5.6.1 section 6.3.1
A smaller production use is to add examples from an actual corpus to a summarization prompt when generic summaries drift from source content. Spot-check the resulting summaries against their source chunks rather than assuming examples alone fix the problem. section 10.6























