A knowledge system becomes easier to update when its documents are kept outside the model’s weights. In retrieval-augmented generation (RAG), modularity means separating retrieval from answer generation: the document repository can change without retraining the whole model. This architectural meaning is distinct from a graph-partition modularity score; no graph metric or calculation is specified here. section 3.4 section 4.7
A common modular implementation, naive RAG, follows four steps:
- Embed the user’s query as a vector. section 4.7
- Retrieve the top-k matching document chunks from a vector database. section 4.7
- Place those chunks and the query in a prompt. section 4.7
- Send the prompt to a pretrained LLM; no joint training is required. section 4.7
This simplicity has a tradeoff. Canonical RAG jointly fine-tunes its retriever and generator and explicitly weighs retrieved documents. Naive RAG uses off-the-shelf components and lets the LLM integrate passages from its prompt. It costs less engineering effort and compute, but may give up gains from task-specific fine-tuning. A reported case study found that a frozen-retriever, trainable-generator arrangement outperformed end-to-end systems costing 100 times more to train; that result is a reason to evaluate simpler designs, not a guarantee for every task. section 4.5.6 section 4.7
Table 4.2 Canonical RAG vs naive RAG
|
Feature
|
Canonical RAG
(Lewis et al., 2020)
|
Naive RAG (Industry Standard)
|
|
Definition
|
The original research involved sophisticated joint
training and explicit probabilistic marginalization over retrieved documents.
|
A simplified, practical pattern using off-the-shelf
components, relying on the LLM's inherent capabilities to integrate retrieved
context.
|
|
Training
|
Jointly fine-tuned the retriever and generator for the
specific task.
|
No joint training; uses a pretrained retriever and LLM.
|
|
Document Fusion
|
Employs explicit marginalization (e.g., RAG-Sequence,
RAG-Token) to weigh document contributions.
|
Concatenates retrieved documents into the LLM's prompt;
relies on the LLM's attention to handle context.
|
|
Complexity
|
Higher architectural and computational complexity requires
specialized training.
|
Much simpler to implement and deploy; significantly lower
computational cost.
|
|
Legacy
|
Established the fundamental retrieval-augmented generation
pattern that subsequent work has refined but not replaced.
|
The default architecture for most real-world RAG
applications is due to its accessibility and effectiveness.
|
Compare canonical and naive RAG by training, document fusion, and implementation complexity. section 4.7
