liveBook · Manning

Approximate nearest neighbor search

As explained in LLM Customization and Fine-Tuning

An indexing/search approach that trades a small amount of recall for much faster nearest-neighbor retrieval.

Approximate nearest-neighbor (ANN) algorithms speed up top-k similarity search by accepting a small loss in recall. Compared with exhaustive search, they can make queries two or three orders of magnitude faster. section 4.5.3

Choose an index approach to match corpus scale and deployment needs:

  • Use a flat NumPy array for prototypes with tens of thousands of chunks. section 4.5.3
  • Consider an in-process library such as FAISS for hundreds of thousands of chunks; HNSW is a common default, while IVF suits corpora sharded across many machines. section 4.5.3
  • For multi-tenant production with millions of chunks and audit logging, use a vector database such as Pinecone, Weaviate, Qdrant, or Postgres with pgvector. section 4.5.3

How LLM Customization and Fine-Tuning covers it

How deep it goes
Defines the recall-speed tradeoff and names HNSW and IVF, but does not detail their internals.
Examples
Compares NumPy, FAISS, and vector databases across corpus sizes and names HNSW and IVF deployment contexts.
What you can do after
Apply the scale-based guidance to choose an index library or vector database for retrieval.
How it connects
Connects ANN to vector indexes, HNSW, IVF, FAISS, and vector databases.

Also explained in

Compare how 7 books explain approximate nearest neighbor search →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.