Similarity search finds relevant passages by comparing representations of a user query with stored document representations. In retrieval-augmented generation (RAG), a retriever selects passages from a vector database; a language model then uses those passages as context for its answer. This supplies information at inference time without changing the model’s weights. section 5.4 section 5.4.3
A typical retrieval path has four stages:
- Chunk documents, encode them with an embedding model, and store the resulting vectors in a vector database. section 5.4.3
- Encode the incoming query and retrieve top-matching passages based on similarity. section 5.4.3 section 3.4
- Optionally rerank or compress the retrieved passages before passing the selected context and original query to the generator. section 5.4.3
Retrieval quality depends on more than the search step. The embedding model must represent semantic meaning well, while chunk size, the number of results retrieved (top-k), and metadata affect what can be found. Exact match, hybrid BM25-plus-vector retrieval, and rerankers are design options. Poor chunking, missing metadata, or ineffective reranking can leave the generator with inadequate context. section 5.4.3
Figure 5.2 Overview of a RAG system. In the preproduction phase, documents from a knowledge base are chunked and embedded into a vector database. During the production phase, a user query is embedded and used to retrieve top-matching documents based on similarity. An optional reranker refines these results to improve relevance. The final ranked context is then passed, together with the original query, to the language model for response generation.
Follow the diagram from document chunking and embedding through query-time matching, optional reranking, and response generation. section 5.4.3
Listing 5.24 Adding a reranker to your RAG system with Flashrank
compressor = FlashrankRerank() #1
compression_retriever = ContextualCompressionRetriever( #2
base_compressor=compressor,
base_retriever=retriever
)
compressed_docs = compression_retriever.invoke( #3
"What was Amazon's net income in 2023?"
)
print([doc.metadata["id"] for doc in compressed_docs])
chain = RetrievalQA.from_chain_type( #4
llm=llm,
retriever=compression_retriever
)
chain.invoke("What was Amazon's net income in 2023?") #5
The Flashrank example wraps an existing retriever in a contextual-compression retriever, checks the returned document IDs for a financial question, and supplies that retriever to a RetrievalQA chain. section 5.4.3
