A support assistant can give a fluent answer that contradicts the policy page it links to. Having the right document within reach is not enough: the answer must use it correctly. Retrieval-augmented generation (RAG) connects a language model to external knowledge so it can synthesize an answer from relevant evidence rather than rely only on its training data. It addresses changing facts, unverifiable claims, and knowledge held outside the model. section 1.0 section 1.3
RAG has two pipelines. Offline, an indexing pipeline splits documents into chunks, converts them into searchable representations, and stores them. At query time, the system processes the question, retrieves candidate chunks, ranks them, and generates an answer using the selected evidence. These stages fail differently: retrieval may miss the needed passage, while generation may ignore or misstate a passage it receives. section 1.2
Warranty question
Without generation, a search system might hand a customer ten warranty PDFs to read. A standalone model might confidently quote terms for the wrong product or an outdated policy. RAG instead retrieves the relevant warranty passages and uses them to compose the response. The passages should also be available for checking: a plausible answer is not proof that its claims are supported. section 1.2 section 1.3.5
The simplest version, often called naive RAG, embeds document chunks, retrieves the most similar ones, and puts them in a model prompt. It is quick to prototype and easier to debug, especially when questions use the same vocabulary as well-structured documents. It can fail when wording differs, sources are noisy, or the answer requires several steps. Try corpus curation, metadata filtering, or retrieval changes against an observed failure before adding architectural complexity. section 1.5.1
Canonical RAG is more specific than the common prompt-based shortcut. The 2020 architecture paired a pretrained dense passage retriever with a BART generator and weighed possible answers across several retrieved passages. In RAG-Sequence, a passage supports an entire generated answer; in RAG-Token, different passages can contribute at different token positions. The latter can help when evidence is split across passages, but its decoding behavior differs. Neither should be confused with simply placing the top chunks in one prompt. section 4.1 section 4.3.3
Listing 4.1 Where the sum over documents goes.
import numpy as np
def marginalize(p_z, p_tok):
"""p(y|x) under sequence-level and token-level marginalization."""
seq = np.sum(p_z * np.prod(p_tok, axis=1)) #A
tok = np.prod(np.sum(p_z[:, None] * p_tok, axis=0)) #B
return float(seq), float(tok)
p_z = np.array([0.55, 0.30, 0.15]) #C
split = np.array([[0.80, 0.10], # names the city, not the state
[0.10, 0.85], # names the state, not the city
[0.20, 0.20]])
whole = np.array([[0.80, 0.75], # one passage carries both
[0.10, 0.10],
[0.20, 0.20]])
for label, p_tok in [("evidence split", split),
("one passage has it all", whole)]:
seq, tok = marginalize(p_z, p_tok)
print(f"{label:22s} sequence={seq:.4f} token={tok:.4f}")The short calculation shows where each variant combines document probabilities: across complete sequences or separately at each token. section 4.3.3
Choose complexity by failure mode. Advanced RAG improves retrieval and context processing; modular RAG adds planned routing, retries, and critique; agentic RAG lets the model choose its next step while answering. More machinery is not automatically better. Measure retrieval and answer quality separately, retain evidence for verification, and account for latency and operational cost. For private knowledge, retrieval also needs appropriate access controls: putting confidential information in a knowledge base does not make every user entitled to receive it. section 1.6 section 1.3.3













































