A vector database supports similarity-based retrieval over embeddings: numerical representations of content. In a retrieval-augmented generation (RAG) system, it helps find document passages that match a question’s meaning, even when the wording differs. For example, a question about updating API credentials can retrieve guidance about key rotation without relying on shared keywords. section 12.1.3
The retrieval path has two phases:
- At build time, process documents into chunks, embed them, and index their vectors. Treat the index, chunking strategy, metadata schema, and preprocessing pipeline as deployment artifacts. section 12.1.3 section 12.1.2
- At query time, embed the question into the same vector space, retrieve similar chunks, and pass their content to the LLM as context for its answer. section 12.1.3
A FAISS-backed RAG path
With LangChain, `vector_store = FAISS.from_documents(documents, embeddings)` creates a store from documents and an embedding model. A retriever can then supply relevant documents to a `RetrievalQA` chain. The equivalent manual path embeds the query, calls `vector_db.similarity_search(query_embedding, k=5)`, joins the returned content into a prompt, and generates an answer. FAISS is presented as a practical choice for development and moderate-scale production; dedicated vector databases such as Pinecone, Weaviate, or Chroma may be needed for distributed operation, real-time changes, metadata filtering, and backup and recovery. section 12.1.3
Figure 12.5 Build-time setup (top) ingests documents into a vector database and configures the LLM. Query-time flow (middle) handles user questions through embedding, retrieval, augmentation, and generation. Implementation requires three lines of LangChain code.
The diagram separates document ingestion and index setup from the query-time embedding, retrieval, augmentation, and generation path. section 12.2.1
In production, the vector store is one component of an LLM application, not a replacement for the ML platform. Existing deployment automation and infrastructure still apply; retrieval adds artifacts and settings such as vector indexes, embeddings, the number of chunks to return, and similarity thresholds. section 12.1.2
