A vector database stores embeddings alongside the content they represent, such as document chunks, so an application can retrieve content by meaning rather than exact wording. An embedding is a numerical vector; similarity search compares a query embedding with stored embeddings and returns the closest matches. This makes a vector database useful when a Q&A chatbot must search many documents instead of sending every document to an LLM. section 6.2.1 section 6.4
Figure 6.3 RAG content ingestion stage. Documents are extracted from sources, split into chunks, and converted into embeddings while being stored in a vector database, which stores a copy of the original chunks and their embeddings (vector form).
Follow the ingestion path: extract documents, split them into chunks, create embeddings, then store each chunk with its vector representation. section 6.1.3
A practical retrieval-augmented generation (RAG) workflow has two stages:
- Index content: load documents, split them into focused chunks, create embeddings, and add the chunks and embeddings to the database. Some stores can create the embeddings when documents are added. section 6.1.3 section 7.1.1
- Answer a question: embed it with a consistent embedding model, retrieve similar chunks, and place those chunks and the question in a prompt for the LLM. section 7.1.2
Search across documents with Chroma
A LangChain setup uses RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0), OpenAIEmbeddings, and Chroma("tourist_info", embeddings_model). Load and split Wikipedia, Word, PDF, or text documents, then call vector_db.add_documents(chunks). To inspect retrieval before generating an answer, call vector_db.similarity_search(query, 4). Returned Document objects include text and source metadata, making it possible to check which sources supplied the matches. section 7.2.1 section 7.3.1
A vector database is more than an early-style vector library. Libraries such as FAISS primarily provide in-memory embedding search and can require separate storage for the original content. Vector databases can store content, embeddings, and metadata together; they also support updates and searches during import. Choose a store with scale, latency, and deployment constraints in mind. section 6.2.3 section 6.4
Treat retrieval quality as part of answer quality. Poorly split chunks can lose context, and an irrelevant match will not reliably ground an answer. Inspect returned chunks, test how many results to retrieve, and keep embedding dimensions consistent between indexing and querying; changing embedding models may require re-embedding the corpus. section 6.4 section 7.3
