liveBook · Manning

Retrieval index

As explained in Retrieval Augmented Generation, The Foundational Ideas

A document index is a stored structure containing representations of documents that supports retrieval from a document collection.

A retrieval index stores searchable representations of documents outside a language model’s weights. In retrieval-augmented generation (RAG), a retriever searches that index and passes selected passages to a generator. Updating the indexed documents can update the system’s available knowledge without retraining the generator. section 4.1.3 section 4.6

A basic PDF question-answering pipeline separates offline indexing from answering a question:

  • Load the PDF, split it into passages, embed each passage, and store the vectors in a searchable index. section 4.6
  • For each question, search the index, put the returned passages into a prompt, and generate an answer from that context. section 4.6
Listing 4.2 Turning a paper into a searchable index.
from langchain_community.document_loaders import PyPDFLoader 
from langchain_text_splitters import RecursiveCharacterTextSplitter 
from langchain_huggingface import HuggingFaceEmbeddings 
from langchain_community.vectorstores import FAISS
PAPER = "https://arxiv.org/pdf/2005.11401" 
splitter = RecursiveCharacterTextSplitter(
  chunk_size=512, chunk_overlap=50
)  #A
chunks = PyPDFLoader(PAPER).load_and_split(text_splitter=splitter) 
embeddings = HuggingFaceEmbeddings(  #B
  model_name="sentence-transformers/all-mpnet-base-v2"
)
index = FAISS.from_documents(chunks, embeddings)  #C
print(f"{len(chunks)} chunks indexed")

The indexing code turns a PDF into 155 chunks and stores their embeddings in FAISS. Replacing the paper means rebuilding this index, not training new model weights. section 4.6

An index can exist and still miss the answer

Asked for the retriever named in the indexed paper, a top-three search returned pages 17, 7, and 18 rather than the introductory passage naming DPR. The generator correctly reported that its context lacked the answer. The failure was retrieval selection, not an absent fact in the PDF: matching passages about retrieval and the top-three cutoff kept the needed passage out. section 4.6

Index replacement and index maintenance are different concerns. In a RAG experiment, a fine-tuned retriever–generator system used a different Wikipedia index at test time and answered with updated information without retraining the generator. In Atlas training, however, changing document embeddings made a prebuilt index stale; recomputing it every 1,000 steps added roughly 30% overhead compared with training with a fixed retriever. Choose an update strategy with both the required freshness and the cost of rebuilding the index in mind. section 4.1.3 section 6.2

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Works through PDF chunking, vector indexing, and query-time search, then examines retrieval misses and training-time index staleness.
Examples
Uses a 155-chunk PDF index, a missed top-three result, a Wikipedia index swap, and Atlas index recomputation.
What it assumes
Introduces loading, chunking, embedding, and indexing from a PDF; uses an initialized language model for answer generation.
What you can do after
Build it from scratch with PDF chunks, embeddings, and FAISS, then diagnose whether missing answers were excluded by retrieval.
How it connects
Connects retrieval-augmented generation, dense embeddings, retrievers, generators, and retriever training.

Also explained in

Compare how 4 books explain retrieval index →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.