Retrieval-augmented generation (RAG) gives an LLM relevant external documents at prompt time. Instead of relying only on training data, a wrapper searches a document database, adds matching content to the prompt, and lets the LLM use it when generating an answer. section 1.7
A practical use
A business chatbot can search internal documents before answering an employee’s question. An application can use the same pattern to analyze up-to-date webpages and identify the sources behind its response. section 1.7
The basic process is straightforward, but the difficult parts are retrieval and prompt size:
- Search for documents relevant to the user’s prompt, often by extracting keywords and matching them against indexed content. section 1.7
- Insert the retrieved documents into the prompt so the LLM can use their contents as additional context. section 1.7
- Keep the added material within the model’s maximum prompt length; longer inputs can also increase usage costs. section 1.7
- Do not treat RAG as magic: irrelevant retrieval produces weak context, and choosing effective retrieval techniques remains an active challenge. section 1.7
RAG changes the prompt rather than the model’s internal parameters. That makes it a distinct alternative to fine-tuning, while increasingly large context windows make it more effective for supplying substantial task-specific information. section 3.3
