liveBook · Manning

Modular RAG

As explained in Retrieval Augmented Generation, The Foundational Ideas

Modular RAG is a retrieval-augmented generation architecture composed of interchangeable specialized components that can be recombined or reconfigured.

A single retrieve-then-generate pipeline can struggle when questions require different evidence sources or several kinds of processing. Modular retrieval-augmented generation (RAG) replaces that fixed pipeline with interchangeable components: retrievers, query processors, fusers, rerankers, and generators can be combined for the job at hand. section 1.5.3 section 8.4

The defining boundary is who controls the workflow. In modular RAG, engineers specify the possible paths in advance; query-time signals can select a route, trigger a retry, or invoke a critique step. In agentic RAG, the model chooses its next step during the run, so the sequence and number of steps can change as it discovers information. A branched flowchart is still modular if every possible path was designed beforehand. section 1.5 section 1.6

Components can be specialized along several axes:

  • Retrieval: use vector search, keyword search, graph queries, or API calls for different information needs. section 1.5.3
  • Processing: handle text, images, or structured data, or select functions such as summarization and fact verification. section 1.5.3
  • Deployment: separate memory-heavy search from compute-heavy generation, and scale each independently behind standardized interfaces. section 1.5.3

Multi-query retrieval as a modular pipeline

RAG-Fusion divides retrieval into query generation, parallel retrieval, and rank fusion. A user question becomes several query variants; each returns a ranked document list; reciprocal rank fusion (RRF) merges those lists before the selected documents become answer context. The parts can be substituted without redesigning the entire pipeline—for example, changing a retriever or adding query expansion. This is a modular composition even when a particular configuration runs the same sequence for every query. section 8.4

Figure 8.3 RAG-Fusion complete data flow.

Follow the query variants through parallel retrieval and rank fusion to see where each component acts. section 8.4

Choose this architecture when distinct query types need specialized strategies, questions draw on multiple sources, or quality control must be systematic. Modularity adds capability and complexity; it is not an automatic upgrade for a proof of concept. Evaluate the failures and costs of the simpler pipeline before adding routes or components. Unlike an agentic workflow, a predefined modular graph also makes its possible paths—and their costs—easier to bound. section 1.5.5 section 1.5 section 15.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains component choices, predefined routing, and a multi-query data flow, but does not specify a universal module interface.
Examples
Traces RAG-Fusion from generated query variants through parallel retrieval and reciprocal rank fusion to answer context.
What it assumes
Assumes familiarity with basic retrieve-then-generate RAG while defining what modularity adds to that pipeline.
What you can do after
Evaluate/understand when to replace a simpler RAG pipeline with predefined routes and specialized, replaceable components.
How it connects
Connects retrieval-augmented generation with vector search, keyword search, graph queries, reciprocal rank fusion, and agentic RAG.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.