chapter ten

10 RAPTOR: Recursive abstractive processing for tree-organized retrieval

 

This chapter covers

  • The Semantic Flattening problem and retrieval failures
  • The RAPTOR architectural pattern
  • Soft clustering with Uniform Manifold Approximation and Projection (UMAP) and Gaussian Mixture Models (GMM)
  • Building a recursive summarization engine
  • Indexing and retrieving strategies for multi-layered trees

Imagine you are a historian tasked with writing a biography of a complex political figure. You enter the national archives expecting to find a structured system: boxes of diaries, organized correspondence, chronological reports, and thematic dossiers. Instead, you find a warehouse floor covered in millions of scraps of paper. Every document has been shredded into individual paragraphs, scattered randomly, stripped of folders, dates, and context.

If you need to answer, "What specific date was the armistice signed?" you might succeed. You can sift through the pile, scan for keywords like "armistice" and "signed," find the relevant scrap, and retrieve the fact. This is how naive RAG works: finding needles in haystacks.

10.1 The semantic flattening problem

10.1.1 Lost in the middle

10.1.2 The limits of fixed-size chunking

10.2 From lists to trees

10.2.1 The recursive mechanism

10.2.2 Visualizing the tree construction

10.3 Clustering in high dimensions

10.3.1 The curse of dimensionality and UMAP

10.3.2 Gaussian Mixture Models and soft clustering

10.3.3 Picking the number of clusters with Bayesian Information Criterion (BIC)

10.4 Practical implementation

10.4.1 The RaptorNode architecture

10.4.2 The clustering engine

10.4.3 The summarization engine

10.4.4 The recursive tree builder

10.5 Indexing and retrieval strategies

10.5.1 Collapsed tree (flattened index)

10.5.2 Implementing tree traversal

10.6 Advanced summarization strategies

10.7 Evaluation, performance, and legacy

10.7.1 Benchmark performance

10.7.2 Operational challenges

10.7.3 Solutions and successors: related architectures

10.8 Summary