The question-answering stage is the retrieval-and-generation phase of retrieval-augmented generation (RAG). A chatbot receives a user question, retrieves relevant content from a vector store or other content store, and sends the question with that context to an LLM for answer generation. section 6.1.3
Build a reliable Q&A workflow by addressing failure points in sequence:
- Transform unclear questions into clearer, more detailed queries before retrieval. section 8.1.2
- Break broad or abstract questions into specific subquestions so retrieval can target more precise content. section 8.1.2
- Route queries to suitable content stores, including vector, relational, table, or graph databases. section 8.1.2
- Generate structured queries such as SQL when relational or graph data cannot process natural-language questions directly. section 8.1.2
- Filter or postprocess retrieved results before passing them to the LLM, reducing irrelevant context and answer noise. section 8.1.2
From question to augmented answer
A chatbot takes a user question, searches indexed content for relevant chunks, and supplies those chunks with the question to an LLM. For broad questions, it can first create narrower subquestions; for structured information, it can generate a database query instead. section 6.1.3 section 8.1.2
Figure 8.2 Common issues in the Q&A stage of the naive RAG architecture and their solutions: (1) Handle poorly formulated questions with question transformation, (2) enhance retrieval accuracy by transforming the original user questions into more suitable vector DB search queries, (3) include relevant data sources by adding structured data content stores, such as relational DBs, (4) generate DB queries for structured data content stores, (5) and filter out irrelevant context retrieved from the content stores.
Use the workflow diagram to trace common naive-RAG failures from question formulation through retrieval and context filtering. section 8.1.2
