A generator turns a question and retrieved passages into a natural-language answer. Its job is synthesis, not search: the retriever finds relevant evidence, and the generator combines it into a response. This can answer questions requiring multiple facts rather than copying a single text span. section 4.0 section 4.6
In canonical RAG, the generator is a pretrained BART sequence-to-sequence model, paired with a pretrained dense passage retriever. Fine-tuning updates the generator and the retriever’s query encoder while leaving the document encoder and its index fixed. The generator learns from question–answer pairs with retrieved passages in its input, encouraging it to use those passages and refrain from unsupported answers. section 4.1 section 4.4
A simpler, naive RAG pipeline instead places retrieved chunks and the question in a prompt to an off-the-shelf chat model. It can work without fine-tuning, but asking a model to rely on context is less reliable than training it to do so. Neither approach can supply evidence that retrieval failed to find. section 4.4 section 4.6
Answer only from retrieved context
For a question about a technical paper, retrieve three document chunks, join their text, and send them with the question to a chat model. The prompt says to answer only from the context and to say when the answer is absent. One example returns the paper’s two RAG formulations; another declines to name its retriever because the retrieved pages do not contain that answer. Returning page numbers helps inspect what the generator received. section 4.6
Listing 4.3 Retrieving, then generating from what was retrieved.
from langchain_core.prompts import ChatPromptTemplate
from providers import get_chat_model
llm = get_chat_model(temperature=0) #A
prompt = ChatPromptTemplate.from_template(
"Answer from the context only. If the context does not "
"contain the answer, say so.\n\n{context}\n\n"
"Question: {question}"
) #B
chain = prompt | llm #C
def answer(question, k=3):
docs = index.similarity_search(question, k=k) #D
context = "\n\n".join(d.page_content for d in docs)
reply = chain.invoke({"context": context, "question": question})
pages = [d.metadata["page"] for d in docs] #E
return reply.content, pagesThe Python listing shows the generator boundary: similarity search supplies passages, a prompt supplies the grounding instruction, and the chat model produces the answer. section 4.6












