liveBook · Manning

Vector index

As explained in LLM Customization and Fine-Tuning

An index structure that stores vectors and enables fast nearest-neighbor or similarity search.

A vector index stores chunk embeddings and makes top-k similarity search fast. In a retrieval-augmented generation pipeline, it connects document ingestion to query-time retrieval: chunks are embedded and indexed once, then a retriever searches the index for each request. section 4.5.2 section 4.5.3

Choose an index approach by corpus scale and deployment needs:

  • For tens of thousands of chunks, a flat NumPy array can be sufficient, but search scales linearly with corpus size. section 4.5.3
  • For hundreds of thousands of chunks, consider an in-process library such as FAISS. Approximate nearest-neighbor methods trade a small amount of recall for two or three orders of magnitude faster queries. section 4.5.3
  • HNSW is a common modern default; IVF can work better when a corpus is sharded across many machines. For multi-tenant production with millions of chunks and audit logging, consider a vector database. section 4.5.3 section 4.5.3

An index is distinct from the operational layer around it: a vector database can add a query API, metadata filters, persistence, replication, multi-tenancy, and observability. If the embedding model changes, re-encode every corpus chunk; embeddings from different models are not directly comparable. section 4.5.3 section 4.5.3

Figure 4.5 Minimal RAG architecture: the ingestion path (chunker, embedder, vector index) runs once at index time; the query path (retriever, prompt assembly, LLM) runs on every request, and both paths meet at the shared vector index.

Follow how the ingestion and query paths meet at the shared vector index. section 4.5.2

How LLM Customization and Fine-Tuning covers it

How deep it goes
Defines the index, compares flat and approximate search, and gives scale-based choices without detailing algorithm internals.
Examples
Compares a NumPy array, FAISS, HNSW, IVF, and vector databases across corpus sizes and deployment needs.
What you can do after
Evaluate index options by corpus size, query speed, recall, and operational requirements.
How it connects
Connects embeddings, retrieval-augmented generation, approximate nearest-neighbor search, and vector databases.

Also explained in

Compare how 11 books explain vector index →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.