A vector database is a specialized database for giving an LLM access to knowledge outside its pretrained information. Instead of storing a document as one large item, it stores smaller text chunks together with embeddings: numeric vectors that represent those chunks. Similar meanings land near one another in embedding space, making the database useful for finding relevant material even when a question does not use the same words as the document. section 2.2 section 4.5
Think of it as a searchable library prepared for meaning rather than exact keyword matches. During retrieval, a user question is represented in the same kind of vector space and compared with stored embeddings. The closest chunks are selected and passed to the LLM as extra context; the LLM then generates an answer from that retrieved knowledge. section 4.1 section 4.5
Figure 4.1 A RAG-based flow includes three essential steps: document ingestion, document retrieval, and answer generation.
This lifecycle diagram places the vector database between document ingestion and runtime retrieval, before answer generation. section 4.1
A typical retrieval-augmented generation workflow has three stages:
- Ingest documents: collect knowledge from files, pages, folders, or databases and transform it into usable text. section 4.1
- Prepare and store chunks: split text into smaller pieces, create embeddings with an embedding model, and save them in the vector database. section 4.2
- Retrieve and answer: search for the few most relevant chunks when a question arrives, then provide them to an LLM to form a response. section 4.1
The timing matters. Ingestion is an offline setup task: run it when the document collection is created or changed. Retrieval and generation are online tasks: run them for every user question. Keeping these stages separate makes a RAG application easier to build, test, and update. section 4.1 section 4.2
A restaurant assistant
A restaurant PDF can be read, split into chunks, embedded, and stored in a vector database. When a customer asks about opening hours, delivery, menu items, or prices, the application searches for the most relevant chunks and gives them to the LLM before it replies. Rebuild the database only when the restaurant information changes. section 4.2 section 4.5
Retrieval quality depends on preparation and search choices. Chunk size, chunk overlap, and the top-k setting—the number of retrieved results—can be adjusted to improve performance. Chunking should also respect meaningful document structure where possible: a cookbook, for example, can be split one recipe at a time rather than across arbitrary text boundaries. section 4.5 section 6.3
A vector database can also serve an agent as a focused knowledge tool. A recipe-searching agent can use semantic search over a cookbook database to find relevant recipes or passages from a desired dish or available ingredients, rather than relying on simple keyword matching. section 6.3
