liveBook · Manning

Hierarchical summarization

As explained in Retrieval Augmented Generation, The Foundational Ideas

Hierarchical summarization is summarization performed across multiple levels of organization, from granular items to broader aggregates.

A flat retrieval index can find a paragraph containing an armistice date yet struggle to answer how a political figure’s foreign policy evolved. The broader answer spans speeches, memos, and drafts; no single chunk contains the whole story. Hierarchical summarization preserves that wider view by producing summaries at successive levels of detail rather than treating every chunk as an isolated search result. section 10.0

In a RAPTOR-style tree, the build starts with raw chunks as leaves. Embed and cluster them, summarize each cluster, then cluster the resulting summaries and repeat. Each new summary links to its children, so retrieval can work with broad themes while retaining a path toward supporting detail. The build stops when one cluster remains or a maximum depth is reached; a separate flat registry can index every node without reconstructing the tree. section 10.4.4

From chunks to a root summary

Suppose 200 leaf chunks form 20 clusters. Summarizing each cluster yields 20 nodes; another pass might reduce those to four summaries, then one root. This bottom-up build happens before queries. The level-keyed tree supports later traversal, while the flat registry supports indexing nodes from all levels in one pass. section 10.4.4

Listing 10.5 Seeding the tree with leaves.
from providers import get_embeddings
class RaptorTreeBuilder:
    def __init__(self, clusterer, summarizer, max_depth=5):
        self.clusterer = clusterer
        self.summarizer = summarizer
        self.max_depth = max_depth  #A
        self.embed_model = get_embeddings()
        self.all_nodes: Dict[str, RaptorNode] = {}  #B
    def build(self, leaf_texts: List[str]):
        embeddings = self.embed_model.embed_documents(leaf_texts)
        level_0 = []
        for text, emb in zip(leaf_texts, embeddings):
            node = RaptorNode(text=text, level=0, embedding=emb)
            self.all_nodes[node.id] = node
            level_0.append(node)
        tree = {0: level_0}
        self._recursive_build(level_0, 1, tree)  #C
        return tree
 

The builder seeds level 0 with embedded text chunks, records each leaf, and starts the recursive build at level 1. section 10.4.4

The hierarchy need not come from text-vector clusters. In graph-based RAG, Leiden communities group related entities: small communities are summarized from their node descriptions, and higher-level communities are summarized from those summaries. For a large litigation file, another pattern starts with grounded retrieval by month, compresses retrieved chunks into month-level summaries, then aggregates them into year-level summaries. Both approaches replace an unwieldy collection of evidence with progressively broader context, but organize that evidence differently. section 12.2.1 section 13.6

Summaries can lose the details a later answer needs. Check candidate summaries against their leaves for dropped entities, and compare faithfulness through retrieval evaluation. Cluster size also matters: a large cluster can make a summarization call slow, expensive, and less effective. Re-cluster oversized groups into sub-batches before summarizing, or cap cluster sizes during the build. Use the extra hierarchy when questions require synthesis across evidence, not simply because a deeper tree is possible. section 10.6 section 10.0

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Works through a bottom-up tree build, then discusses graph-community aggregation and oversized-cluster handling; leaves summary-model selection to evaluation.
Examples
Uses a 200-chunk recursive build, Leiden community summaries, and month-to-year synthesis of a litigation file.
What it assumes
Assumes access to embeddings and a clustering engine while starting the worked tree build from raw chunks.
What you can do after
Build it from scratch by clustering chunks, summarizing groups recursively, linking parent and child nodes, and checking summary faithfulness.
How it connects
Links embeddings, clustering, retrieval trees, graph communities, and context compression.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.