A single retrieve-then-generate pipeline can struggle when questions require different evidence sources or several kinds of processing. Modular retrieval-augmented generation (RAG) replaces that fixed pipeline with interchangeable components: retrievers, query processors, fusers, rerankers, and generators can be combined for the job at hand. section 1.5.3 section 8.4
The defining boundary is who controls the workflow. In modular RAG, engineers specify the possible paths in advance; query-time signals can select a route, trigger a retry, or invoke a critique step. In agentic RAG, the model chooses its next step during the run, so the sequence and number of steps can change as it discovers information. A branched flowchart is still modular if every possible path was designed beforehand. section 1.5 section 1.6
Components can be specialized along several axes:
- Retrieval: use vector search, keyword search, graph queries, or API calls for different information needs. section 1.5.3
- Processing: handle text, images, or structured data, or select functions such as summarization and fact verification. section 1.5.3
- Deployment: separate memory-heavy search from compute-heavy generation, and scale each independently behind standardized interfaces. section 1.5.3
Multi-query retrieval as a modular pipeline
RAG-Fusion divides retrieval into query generation, parallel retrieval, and rank fusion. A user question becomes several query variants; each returns a ranked document list; reciprocal rank fusion (RRF) merges those lists before the selected documents become answer context. The parts can be substituted without redesigning the entire pipeline—for example, changing a retriever or adding query expansion. This is a modular composition even when a particular configuration runs the same sequence for every query. section 8.4
Figure 8.3 RAG-Fusion complete data flow.

Follow the query variants through parallel retrieval and rank fusion to see where each component acts. section 8.4
Choose this architecture when distinct query types need specialized strategies, questions draw on multiple sources, or quality control must be systematic. Modularity adds capability and complexity; it is not an automatic upgrade for a proof of concept. Evaluate the failures and costs of the simpler pipeline before adding routes or components. Unlike an agentic workflow, a predefined modular graph also makes its possible paths—and their costs—easier to bound. section 1.5.5 section 1.5 section 15.1
