A Q&A engine answers a client system’s question using information from a knowledge base. Unlike a chatbot, it returns one answer rather than keeping a conversation open for follow-up questions. That makes a missing detail especially costly: the client cannot simply ask the engine to clarify its reply. section 1.1.1 section 6.1.1
A retrieval-augmented Q&A engine has two phases:
- Ingest: pull in source text, split it into chunks, convert the chunks into embeddings, and store both chunks and embeddings in a vector store. section 1.1.1
- Query: embed the question with the same model, retrieve relevant chunks through semantic search, and send those chunks with the question to an LLM to produce an answer. section 1.1.1
Figure 1.2 A Q&A engine implemented with RAG design. An LLM query engine stores domain-specific document information in a vector store. When an external system sends a query, it converts the natural language question into its embeddings (or vector) representation, retrieves the related documents from the vector store, and then gives the LLM the information it needs to craft a natural language response.
The diagram shows how a question travels from an external system through retrieval to an LLM-generated response. section 1.1.1
The cost of an incomplete answer
Asked about the temples at Paestum, a chatbot identified the temples and their architectural style but omitted who built them. A follow-up question recovered that detail. A one-answer Q&A engine would have returned the incomplete response to its client, with no conversational repair step. section 6.1.1
