liveBook · Manning

Modularity

As explained in Retrieval Augmented Generation, The Foundational Ideas

Modularity is a graph-partition quality measure comparing the observed density of within-community edges with a null-model expectation.

A knowledge system becomes easier to update when its documents are kept outside the model’s weights. In retrieval-augmented generation (RAG), modularity means separating retrieval from answer generation: the document repository can change without retraining the whole model. This architectural meaning is distinct from a graph-partition modularity score; no graph metric or calculation is specified here. section 3.4 section 4.7

A common modular implementation, naive RAG, follows four steps:

  • Embed the user’s query as a vector. section 4.7
  • Retrieve the top-k matching document chunks from a vector database. section 4.7
  • Place those chunks and the query in a prompt. section 4.7
  • Send the prompt to a pretrained LLM; no joint training is required. section 4.7

This simplicity has a tradeoff. Canonical RAG jointly fine-tunes its retriever and generator and explicitly weighs retrieved documents. Naive RAG uses off-the-shelf components and lets the LLM integrate passages from its prompt. It costs less engineering effort and compute, but may give up gains from task-specific fine-tuning. A reported case study found that a frozen-retriever, trainable-generator arrangement outperformed end-to-end systems costing 100 times more to train; that result is a reason to evaluate simpler designs, not a guarantee for every task. section 4.5.6 section 4.7

Table 4.2 Canonical RAG vs naive RAG
Feature
Canonical RAG (Lewis et al., 2020)
Naive RAG (Industry Standard)
Definition
The original research involved sophisticated joint training and explicit probabilistic marginalization over retrieved documents.
A simplified, practical pattern using off-the-shelf components, relying on the LLM's inherent capabilities to integrate retrieved context.
Training
Jointly fine-tuned the retriever and generator for the specific task.
No joint training; uses a pretrained retriever and LLM.
Document Fusion
Employs explicit marginalization (e.g., RAG-Sequence, RAG-Token) to weigh document contributions.
Concatenates retrieved documents into the LLM's prompt; relies on the LLM's attention to handle context.
 
Complexity
Higher architectural and computational complexity requires specialized training.
Much simpler to implement and deploy; significantly lower computational cost.
Legacy
Established the fundamental retrieval-augmented generation pattern that subsequent work has refined but not replaced.
The default architecture for most real-world RAG applications is due to its accessibility and effectiveness.

Compare canonical and naive RAG by training, document fusion, and implementation complexity. section 4.7

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains architectural tradeoffs and a four-step pipeline, but provides no graph-partition metric or calculation.
Examples
Contrasts a frozen-retriever case study with query embedding, top-k retrieval, prompt assembly, and an LLM call.
What it assumes
Names vectors, vector databases, prompts, and LLMs without defining those components from zero.
What you can do after
Evaluate/understand when a simpler modular RAG design may justify its cost and performance tradeoffs.
How it connects
Connects modularity to retrieval-augmented generation, fine-tuning, vector databases, and prompt assembly.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.