liveBook · Manning

DPR pattern

As explained in Agent Design Patterns

An agent design pattern whose described components include a Decomposer that dispatches work and a Recomposer that combines partial intermediaries into a single outcome.

A large input can exceed a model’s useful perception window even when it technically fits. Downsampling may discard needed detail; saturated attention may lead to omissions, hallucinations, or inaccurate retrieval. Decompose-Process-Recompose (DPR) addresses this by giving a worker manageable fragments, then combining their results into one answer. section 8.0 section 8.5

A DPR component has three stages:

  • Decompose the input into fragments that fit the processor, preferably preserving enough information to reassemble the input with little or no loss. section 8.2
  • Process each fragment with a worker, producing one intermediate result per fragment. Independent fragments can be processed concurrently; fragments needing global context cannot always be. section 8.2 section 8.2.2
  • Recompose the intermediates into a single result that does not expose the fragmentation. Recomposition may be procedural or use another intelligent component. section 8.2

DPR divides one input rather than making copies for a Voter. Unlike an Orchestrator, it does not manage fragment work beyond dispatch and synchronization. Per-fragment overhead can raise cost, while parallel work can reduce latency; retaining intermediates can improve interpretability. section 8.2.4 section 8.2.1

Chunked text processing

A simple decomposer cuts text into roughly 4,096-character blocks, choosing sentence boundaries where possible. An asynchronous worker processes the blocks, and a recomposer receives the collected results. Map-reduce summarization, multimodal extraction, and sequential translation are further applications. section 8.2.2 section 8.4

Listing 8.1 dpr.py: A high-level code sketch of the Decompose-Process-Recompose pattern
from ..utils import E, IE
import asyncio
 
 
class DPR(IE[[str], str]):
  def __init__(
    self,
    decomposer: E[[str], list[str]],
    processor: IE[[str], str],
    recomposer: IE[[list[str]], str],
  ):
    self.decomposer = decomposer
    self.processor = processor
    self.recomposer = recomposer
 
  async def __call__(self, input: str) -> str:
    # decompose
    decomposed_inputs = await self.decomposer(input)
 
    # process
    processed_inputs = await asyncio.gather(
      *[
        self.processor(input)
        for input in decomposed_inputs
      ]
    )
 
    # recompose
    return await self.recomposer(processed_inputs)

The Python sketch makes the component boundaries explicit: call the decomposer, gather worker results asynchronously, then pass them to the recomposer. section 8.2.2

How Agent Design Patterns covers it

How deep it goes
Defines three stages and their properties, then sketches asynchronous control flow; leaves sophisticated global-context handling unspecified.
Examples
Uses sentence-boundary text chunking and names map-reduce summarization, multimodal extraction, and sequential translation.
What it assumes
Assumes familiarity with Python-like asynchronous callables while defining decomposition, processing, and recomposition explicitly.
What you can do after
Build it from scratch as a decomposer, fragment processor, and recomposer, with asynchronous processing where fragments are independent.
How it connects
Contrasts DPR with Voter and Orchestrator, and names Decompose-Retriever and Retriever-Process-Recompose variants.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Agent Design Patterns. The section links point to the parts of the book it draws on.