Retrieval-augmented generation (RAG) answers a query using relevant external information retrieved before the response is generated. Instead of relying only on an LLM’s pretrained knowledge, it supplies topic-specific context from a document collection, making answers more relevant to the user’s data. section 7.2
Figure 7.8 Architecture of a RAG system. The system first indexes a collection of documents offline into a vector database, enabling efficient semantic search. When a user query is received, the system embeds the query, searches the vector database for relevant documents, and retrieves the most relevant contexts. These contexts are then used to augment the original query, creating an enriched prompt that is passed to a large language model to generate a final response.
Follow the two paths: documents are indexed ahead of time; each incoming query retrieves context that is added to the prompt before generation. section 7.2.2
A document-backed chat application needs four connected stages:
- Index: sanitize and chunk documents, create embeddings, and store the chunks with metadata in a vector store. section 7.2.1 section 7.2.4
- Retrieve: embed the query and rank relevant chunks. A LangChain.js retriever can load a saved HNSWLib index and select six results with `asRetriever({ k: 6 })`. section 7.2.2 section 7.2.4
- Augment and generate: place retrieved context beside the question in a prompt, then pass it to the LLM. Return the answer and source documents to the application. section 7.2.4
Run a document-backed chat app
For the HNSWLib-based Next.js example, set the required API keys, index the `corpus` directory with `node scripts/indexDocuments.google.js -d corpus`, and set `RAG_INDEX_PATH` to the absolute path of the resulting index. Start the app with `npm run dev -w ch07/rag-web-app`. Its server action loads the index once, calls `performRAG(input)` for each query, and displays the answer in the conversation. section 7.2.3 section 7.2.4 section 7.2.5
A prompt such as “Answer the question based only on the context provided” directs generation toward retrieved documents, but retrieval alone is not a guarantee of factual accuracy. Grounding support can add source checks or annotations; an enhanced implementation also records log-probability confidence metrics and can ask the user when confidence falls below a configured threshold. section 7.2.4 section 7.2.6
For multiple knowledge bases, extend the same retrieval-and-generation flow with document uploads, authenticated users, and isolation between their data. One full-stack configuration uses Clerk for authentication and Upstash Redis and Vector for state and search, rather than the filesystem-backed HNSWLib index. section 11.0 section 11.1 section 11.1.2
