Response generation turns retrieved evidence into a natural-language answer. In a warranty-support scenario, returning PDFs leaves the customer to find the answer, while a standalone language model may give outdated or irrelevant terms. RAG instead grounds its response in selected evidence. section 1.2
Generation is the final stage of the real-time RAG query pipeline:
- Process the question, retrieve candidate documents, and rank them to prioritize relevant evidence. section 1.2
- Synthesize a response grounded in the curated evidence, rather than merely returning documents or extracting a span. section 1.2 section 4.5.6
Synthesis matters when retrieved passages must become a coherent answer. A case study found that generating novel answers from evidence, rather than extracting spans, was a decisive advantage. It also reported that a frozen retriever with a trainable generator outperformed end-to-end systems at a fraction of their training cost; this supports a modular design, not a claim that complexity always wins. section 2.4.2 section 4.5.6
Generation can also be coupled more closely to ranking: RankRAG instruction-tunes one model to identify relevant passages and generate answers from them, treating the two abilities as related. section 15.3
Figure 1.1 The RAG workflow: from user queries to grounded responses through retrieval and generation.

Trace how a user query moves through retrieval and generation to a grounded response. section 1.2



