A vector index stores chunk embeddings and makes top-k similarity search fast. In a retrieval-augmented generation pipeline, it connects document ingestion to query-time retrieval: chunks are embedded and indexed once, then a retriever searches the index for each request. section 4.5.2 section 4.5.3
Choose an index approach by corpus scale and deployment needs:
- For tens of thousands of chunks, a flat NumPy array can be sufficient, but search scales linearly with corpus size. section 4.5.3
- For hundreds of thousands of chunks, consider an in-process library such as FAISS. Approximate nearest-neighbor methods trade a small amount of recall for two or three orders of magnitude faster queries. section 4.5.3
- HNSW is a common modern default; IVF can work better when a corpus is sharded across many machines. For multi-tenant production with millions of chunks and audit logging, consider a vector database. section 4.5.3 section 4.5.3
An index is distinct from the operational layer around it: a vector database can add a query API, metadata filters, persistence, replication, multi-tenancy, and observability. If the embedding model changes, re-encode every corpus chunk; embeddings from different models are not directly comparable. section 4.5.3 section 4.5.3
Figure 4.5 Minimal RAG architecture: the ingestion path (chunker, embedder, vector index) runs once at index time; the query path (retriever, prompt assembly, LLM) runs on every request, and both paths meet at the shared vector index.

Follow how the ingestion and query paths meet at the shared vector index. section 4.5.2










