liveBook · Manning

Decompose-then-recompose

As explained in Retrieval Augmented Generation, The Foundational Ideas

Decompose-then-recompose is a context-processing technique that splits retrieved documents into fine-grained segments, filters them, and recombines the useful text.

A relevant retrieved document can still be mostly noise. If a question asks for one transaction limit, passing an entire policy to the generator may bury the answer in boilerplate. Decompose-then-recompose reduces that risk by turning retrieved documents into a shorter, query-focused context. section 11.2.2

The refinement has three steps:

  • Decompose each retrieved document into fine-grained strips, such as sentences or paragraphs. section 11.2.2
  • Re-score each strip against the query and discard irrelevant strips, even when their parent document is relevant. section 11.2.2
  • Concatenate the remaining high-utility strips into the context sent to the generator. section 11.2.2

Extracting a policy limit

For a question about a transaction limit, keep the table row containing the limit and drop surrounding policy boilerplate. The generator receives the pertinent fact rather than the whole document. section 11.2.2

In a CRAG pipeline, a retrieval evaluator first scores documents and decides whether internal results are trustworthy, ambiguous, or unsuitable. The pipeline then refines useful internal results, falls back to web search, or combines sources as appropriate; web results pass through the same refinement. Document-level evaluation decides where to look, while strip-level filtering decides what text to keep. section 11.2.4 section 11.2.2

Listing 11.3 The CRAG orchestrator.
class CRAGPipeline:
    """Orchestrates the retrieve-evaluate-correct loop."""
 
    def __init__(self, retriever, evaluator, web_searcher):
        self.retriever = retriever
        self.evaluator = evaluator
        self.web_searcher = web_searcher
 
    def run(self, query: str) -> dict:
        """Executes the corrective RAG workflow."""
        internal_docs = self.retriever.retrieve(query)  #A
 
        action, score = self.evaluator.evaluate(query, internal_docs)  #B
 
        context = ""
        source = "internal"
 
        if action == RetrievalAction.INCORRECT:  #C
            web_docs = self.web_searcher.search(query)  #D
            context = self.evaluator.refine_knowledge(query, web_docs)
            source = "web"
 
        elif action == RetrievalAction.AMBIGUOUS:
            internal_ctx = self.evaluator.refine_knowledge(  #E
                query, internal_docs)
            web_docs = self.web_searcher.search(query)
            web_ctx = self.evaluator.refine_knowledge(query, web_docs)
 
            context = f"{internal_ctx}\n\n{web_ctx}"  #F
            source = "hybrid"
 
        else:
            context = self.evaluator.refine_knowledge(query, internal_docs)  #G
 
        return {"context": context, "action": action.value,
                "source": source, "score": score}
 
 

The Python orchestrator shows where refinement fits after retrieval evaluation and where a web-search fallback enters the workflow. section 11.2.4

Refinement cannot make stale or missing evidence correct. Keep the fallback decision separate from text filtering, and tune evaluator thresholds on held-out queries rather than treating defaults as universal: an ambiguous result may need supplementary search, not merely a shorter excerpt. section 11.2.4

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Specifies segmentation, strip-level re-scoring, and recomposition, then places refinement in a Python pipeline without prescribing a scoring model.
Examples
Uses a transaction-limit table row within a longer compliance policy and a Python CRAG orchestrator.
What it assumes
Assumes familiarity with queries, retrieved documents, and generator context while defining the refinement steps explicitly.
What you can do after
Operate/configure it in a CRAG pipeline by refining internal or web results and tuning evaluator thresholds on held-out queries.
How it connects
Links retrieval evaluation, corrective RAG, web-search fallback, and lost-in-the-middle errors.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.