Chapter 7: Retriever: Returning more relevant results
Introduces the Retriever pattern as a way to search large information sources cheaply, then give an agent a small, relevant context for reasoning.
After this chapter you will be able to
- Scaling wall of context. Explain why an agent cannot simply give a model every potentially useful record: context windows are finite, and longer inputs increase the cost of model calls.
- Broad search and focused judging. Trace the insurance-claim example from preparing the new claim for search, through a cheap, broad search of historical claims, to closer evaluation of candidates and final adjudication using a few relevant precedents and the official guide. Explain that the broad stage favors recall while the narrower stage improves precision and keeps final reasoning manageable.
- Selective retrieval pain point. Explain why an agent cannot feasibly read an entire large knowledge source and how Retriever addresses that limitation by finding and filtering task-relevant information for the model to see.
- Three-stage retrieval pipeline. Trace a query through search element preparation, search execution, and search result processing, explaining what each stage contributes.
- Retriever selection and sensitivity. Explain the Retriever’s selectivity, meaning-based matching in neural retrieval, and desired response to semantic query changes versus ordinary wording noise.
- Grounding and answer quality. Explain that retrieved facts can ground an agent’s answers, reducing hallucinations and typically improving its capability and interpretability.
- Context efficiency and latency. Describe how retrieving relevant parts of a large input can lower cost by avoiding irrelevant tokens and improve alignment through curated context, while complex re-ranking or query transformation can add latency.
- Retriever pipeline stages. Describe how the generic Retriever class encapsulates query pre-processing and search execution, with optional result post-processing.
- RAG context for generation. Explain how Retriever supports retrieval-augmented generation by fetching context relevant to a user query and injecting it into a Generator IE’s prompt so it can answer questions beyond its training data.
- Retriever roles in larger patterns. Explain how retrievers support Memory by fetching relevant context for an agentic operation, and support large sparse Swarms by finding entry-point agents to invoke.
- Hybrid retrieval design. Describe how a hybrid retriever runs several search algorithms in parallel over the same database and combines their results.
- Complementary search strengths. Explain why a typical hybrid retriever combines dense semantic search with sparse keyword search: semantic matching can find related concepts but miss exact identifiers, while keyword matching finds exact terms but can miss synonyms or intent.
- Resampling retriever feedback loop. Explain how a resampling retriever combines the Retriever and Resampler patterns: it fetches results, evaluates their relevance, and either proceeds or requests an alternative query formulation.
- Grounding support answers with retrieval. Explain why a support assistant retrieves relevant knowledge-base paragraphs before generating an answer: a general-purpose model may lack private documentation, manuals, or current policies, and retrieved context lets it answer specific questions without fine-tuning on those materials.
- Complementary technical support searches. Explain why technical-support queries need both dense and keyword retrieval: dense search handles conceptual descriptions, while keyword search catches exact codes, model numbers, and rare terms.
- Hybrid retrieval and reranking pipeline. Trace how the Hybrid Retriever sends a query to dense vector and sparse keyword indexes in parallel, fuses their results—commonly with Reciprocal Rank Fusion—and uses a cross-encoder re-ranker to select the most relevant documents.
- Dynamic few-shot exemplar selection. Explain why fixed few-shot examples may be poorly matched to a new query and how a Retriever instead selects similar, high-quality input-output examples and inserts them into the prompt.
- Semantic selection of tools. Explain why an agent with hundreds or thousands of tools cannot include every tool description in its system prompt, and describe how a Retriever selects tools by matching the agent’s intent to stored tool descriptions and schemas.
What it builds on
- How the Resampler pattern works (chapter 5) · The Resampler pattern’s use of repeated attempts.
Section by section
Introduction
Introduces the Retriever pattern as a way to search large information sources cheaply, then give an agent a small, relevant context for reasoning.
- Scaling wall of context: Explain why an agent cannot simply give a model every potentially useful record: context windows are finite, and longer inputs increase the cost of model calls.
Why it matters: This constraint explains why retrieval is needed before expensive reasoning. - Broad search and focused judging: Trace the insurance-claim example from preparing the new claim for search, through a cheap, broad search of historical claims, to closer evaluation of candidates and final adjudication using a few relevant precedents and the official guide. Explain that the broad stage favors recall while the narrower stage improves precision and keeps final reasoning manageable.
Why it matters: The two stages let an agent search at scale without spending close-attention costs on every record. - Retrieval beyond document databases: Describe how the same selective look-up approach can retrieve useful past actions, details from a large input, or events in visual and audio records, rather than applying only to a static document database.
Why it matters: Retrieval is a general way to give agents focused access to information they cannot repeatedly process in full.
Key terms: scaling wall of context, context window, computational cost, Retriever pattern, input representation, vector embedding, cheap search, high recall, post-processing, action histories
7.1 The pain point:
This section explains why agents need to select useful information from large external sources rather than process everything in context.
- Selective retrieval pain point: Explain why an agent cannot feasibly read an entire large knowledge source and how Retriever addresses that limitation by finding and filtering task-relevant information for the model to see.
Why it matters: Selecting useful information keeps the model’s input focused while avoiding the cost and delay of processing everything.
Key terms: Retriever, worker IE, effective perception window, context windows, selective attention
7.2 How the Retriever pattern works
This section explains how a Retriever turns a query into a small, relevant selection from a large information pool through preparation, search, and result processing.
- Three-stage retrieval pipeline: Trace a query through search element preparation, search execution, and search result processing, explaining what each stage contributes.
Why it matters: The stages show where a retrieval system can improve relevance or efficiency. - Search representations and refinement: Describe the query representations named for dense, sparse, and late-interaction retrieval, and explain why an efficient initial search can be followed by more precise, costly evaluation.
Why it matters: This tradeoff explains how retrieval can search a large pool without treating every item as an expensive comparison. - Retriever selection and sensitivity: Explain the Retriever’s selectivity, meaning-based matching in neural retrieval, and desired response to semantic query changes versus ordinary wording noise.
Why it matters: These characteristics describe what makes retrieved results useful as an agent’s operational context.
Key terms: Retriever, search element preparation, search execution, search result processing, vector embedding, weighted keyword vector, ColBERT, matrix of token vectors, approximate nearest neighbor (ANN), re-ranking
7.2.1 Tradeoffs
This subsection weighs the gains from retrieval-grounded context against the latency that more complex retrieval steps can add.
- Grounding and answer quality: Explain that retrieved facts can ground an agent’s answers, reducing hallucinations and typically improving its capability and interpretability.
Why it matters: Grounding helps explain why adding retrieval can make an agent’s answers more useful and easier to understand. - Context efficiency and latency: Describe how retrieving relevant parts of a large input can lower cost by avoiding irrelevant tokens and improve alignment through curated context, while complex re-ranking or query transformation can add latency.
Why it matters: These tradeoffs help you decide whether the benefits of selective context outweigh the extra time retrieval may take.
Key terms: Capability, Interpretability, retrieved facts, hallucination, Cost, Alignment, irrelevant tokens, curated context, latency, re-ranking
7.2.2 Implementation sketch
This subsection sketches a generic neural Retriever class and explains which parts it encapsulates, accepts as inputs, or relies on as dependencies.
- Retriever pipeline stages: Describe how the generic Retriever class encapsulates query pre-processing and search execution, with optional result post-processing.
Why it matters: These stages give you a way to explain what the Retriever does when a query arrives. - Retriever inputs and dependencies: Distinguish the query and database, which the abstraction treats as inputs, from embedding, search, and re-ranking processes, which it treats as dependencies. State that production frameworks often package the individual steps into one ready-to-use Retriever utility.
Why it matters: This distinction clarifies the sketch’s interface without assuming you must assemble every step yourself.
Key terms: neural Retriever, query pre-processing, search execution, result post-processing, query, database, embedding, search, re-ranking, ready-to-use utility package
7.2.3 When and where to use Retriever
This section identifies three uses for Retriever: grounding generation with relevant context, recalling an agent’s past interactions, and selecting tools or agents.
- RAG context for generation: Explain how Retriever supports retrieval-augmented generation by fetching context relevant to a user query and injecting it into a Generator IE’s prompt so it can answer questions beyond its training data.
Why it matters: Retrieved context lets a generator address private data, recent events, and specialized technical domains. - Relevant long-term memory: Describe how an agent can store past interactions in vector storage and retrieve semantically relevant conversation history or observations as long-term memory.
Why it matters: This use lets an agent bring pertinent details from earlier interactions into its current context. - Semantic tool and agent selection: Explain how Retriever can select a tool or agent from a large pool by matching a natural-language description of the intended action to descriptions of available tools or agents.
Why it matters: Treating tool descriptions as retrievable items provides a way to find a suitable option among many.
Key terms: retrieval-augmented generation, RAG, Generator IE, context, long-term memory, conversation history, past observations, vector storage, semantically relevant, routing
7.2.4 How Retriever works with other patterns
This section shows how Retriever serves as a component of larger Memory and Swarm patterns.
- Retriever roles in larger patterns: Explain how retrievers support Memory by fetching relevant context for an agentic operation, and support large sparse Swarms by finding entry-point agents to invoke.
Why it matters: The same retrieval mechanism can help an agent find useful context or help a swarm decide where to begin.
Key terms: Retriever, Memory, Swarm, hybrid retrievers, relevant context, entry-point agents
7.3 Retriever pattern variants
The section introduces hybrid retrievers, which combine parallel search strategies to return results that either strategy might miss alone.
- Hybrid retrieval design: Describe how a hybrid retriever runs several search algorithms in parallel over the same database and combines their results.
Why it matters: This is the design that makes hybrid retrieval different from using a single search strategy. - Complementary search strengths: Explain why a typical hybrid retriever combines dense semantic search with sparse keyword search: semantic matching can find related concepts but miss exact identifiers, while keyword matching finds exact terms but can miss synonyms or intent.
Why it matters: The two methods compensate for different kinds of missed results.
Key terms: hybrid retriever, parallel, fuse their results, dense neural retrieval, semantic matching, sparse retrieval, BM25, SPLADE, keyword matching
7.3.1 Resampling retriever
This section explains how a resampling retriever improves relevance by evaluating search results and trying a new query when relevance is low.
- Resampling retriever feedback loop: Explain how a resampling retriever combines the Retriever and Resampler patterns: it fetches results, evaluates their relevance, and either proceeds or requests an alternative query formulation.
Why it matters: The feedback loop lets retrieval try again rather than passively accepting irrelevant results.
Key terms: resampling retriever, active retriever, lightweight evaluator IE, re-sampling
7.4 Illustrative Applying the Retriever pattern: use-case examples
7.4.1 Context-aware assistant
Shows how a support assistant uses retrieved documentation to ground its answers without fine-tuning the language model.
- Grounding support answers with retrieval: Explain why a support assistant retrieves relevant knowledge-base paragraphs before generating an answer: a general-purpose model may lack private documentation, manuals, or current policies, and retrieved context lets it answer specific questions without fine-tuning on those materials.
Why it matters: Retrieval gives the assistant access to information the model may not have learned. - Dense retrieval and reranking: Describe the standard implementation as a dense retriever followed by a cross-encoder re-ranker, with reranking used to improve the precision of the material placed in the context window.
Why it matters: The quality of the selected passages affects how useful the context is for answering.
Key terms: Retriever, retrieval-augmented generation (RAG), retrieved context, grounds, knowledge base, dense retriever, cross-encoder re-ranker, high precision, context window
7.4.2 Hybrid RAG for technical support
This section shows how a hybrid Retriever combines semantic and keyword search to handle both vague technical-support descriptions and exact identifiers.
- Complementary technical support searches: Explain why technical-support queries need both dense and keyword retrieval: dense search handles conceptual descriptions, while keyword search catches exact codes, model numbers, and rare terms.
Why it matters: Choosing only one search method can miss an important class of support queries. - Hybrid retrieval and reranking pipeline: Trace how the Hybrid Retriever sends a query to dense vector and sparse keyword indexes in parallel, fuses their results—commonly with Reciprocal Rank Fusion—and uses a cross-encoder re-ranker to select the most relevant documents.
Why it matters: The order of these steps explains how the system combines broad coverage with a final relevance check.
Key terms: dense retriever, keyword-based search, BM25, semantic intent, Hybrid Retriever, dense vector index, sparse keyword index, Reciprocal Rank Fusion (RRF), cross-encoder re-ranker
7.4.3 Dynamic few-shot exemplar selection
This section shows how a Retriever selects relevant examples for each task and inserts them into a few-shot prompt.
- Dynamic few-shot exemplar selection: Explain why fixed few-shot examples may be poorly matched to a new query and how a Retriever instead selects similar, high-quality input-output examples and inserts them into the prompt.
Why it matters: Choosing examples for the current task can give the model more useful reasoning patterns than using the same examples every time.
Key terms: in-context learning, static few-shot examples, Retriever pattern, dynamic few-shot prompting, training examples
7.4.4 Semantic tool selection
This section shows how the Retriever pattern selects relevant tools by matching an agent’s intent to tool descriptions and schemas without putting the entire toolkit in the prompt.
- Semantic selection of tools: Explain why an agent with hundreds or thousands of tools cannot include every tool description in its system prompt, and describe how a Retriever selects tools by matching the agent’s intent to stored tool descriptions and schemas.
Why it matters: Selecting only relevant tools lets an agent use a large toolkit while keeping its context window lean.
Key terms: Retriever pattern, semantic router, tool descriptions, schemas, semantic similarity, context window
7.5 Summary
The section recaps how Retriever keeps agents efficient by selecting relevant context from large data sources through a staged search pipeline and its variants.
Where this is used later
- Retriever-Process-Recompose (chapter 8) · Applies retrieval when processing large inputs.
Concepts explained in this chapter
- RetrieverAn agent design pattern that retrieves relevant information from a larger data source to provide focused context for downstream processing.
- embeddingA numerical vector representation of data, often designed to encode semantic features for computational comparison.
- lexical searchKeyword-based search retrieves information by matching query terms against terms in the searchable data.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
