liveBook · Manning

Separation of concerns

As explained in Retrieval Augmented Generation, The Foundational Ideas

Separation of concerns is a design principle that keeps distinct responsibilities in separate system components.

Separation of concerns keeps knowledge storage and reasoning in distinct components. In retrieval-augmented generation, a retriever finds information while a generator synthesizes it, improving maintainability, updateability, and comprehensibility. section 4.7

The pattern provides practical architectural benefits:

  • Specialized components can collectively outperform a monolithic system when each is effective at its own task. section 4.7
  • Independent knowledge storage enables updates and real-time adaptation without retraining the entire system. section 4.7
  • Simple retrieve-and-generate pipelines favor deployment and adoption over the complexity of joint training. section 4.7
Table 4.2 Canonical RAG vs naive RAG
Feature
Canonical RAG (Lewis et al., 2020)
Naive RAG (Industry Standard)
Definition
The original research involved sophisticated joint training and explicit probabilistic marginalization over retrieved documents.
A simplified, practical pattern using off-the-shelf components, relying on the LLM's inherent capabilities to integrate retrieved context.
Training
Jointly fine-tuned the retriever and generator for the specific task.
No joint training; uses a pretrained retriever and LLM.
Document Fusion
Employs explicit marginalization (e.g., RAG-Sequence, RAG-Token) to weigh document contributions.
Concatenates retrieved documents into the LLM's prompt; relies on the LLM's attention to handle context.
 
Complexity
Higher architectural and computational complexity requires specialized training.
Much simpler to implement and deploy; significantly lower computational cost.
Legacy
Established the fundamental retrieval-augmented generation pattern that subsequent work has refined but not replaced.
The default architecture for most real-world RAG applications is due to its accessibility and effectiveness.

Compare canonical RAG's jointly trained, explicitly marginalized architecture with naive RAG's off-the-shelf components and prompt-based document fusion. section 4.7

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains component responsibilities and contrasts joint training and marginalization with simpler prompt-based document fusion.
Examples
Uses retriever-generator pipelines, retrieved-document concatenation, and hot-swappable knowledge as examples.
What you can do after
Evaluate RAG designs by weighing modularity, updateability, implementation simplicity, and architectural complexity.
How it connects
Connects retrieval, generation, RAG, pretrained components, vector databases, and LangChain.

Also explained in

Compare how 25 books explain separation of concerns →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.