A hierarchical index organizes retrieval representations as a tree-like structure instead of a flat list of vectors. It addresses semantic flattening, incorrect specificity, and incomplete answers by preserving relationships between detailed passages and broader summaries. section 10.0
A RAPTOR-style index contains multiple retrieval resolutions:
- Leaf nodes retain original text chunks for specific facts, such as a support phone number or contract fee. section 10.2
- Summary nodes group semantically related chunks and store LLM-generated abstractions, such as a summary of a termination clause. section 10.2
- Higher layers recursively summarize lower layers until a root represents the entire corpus. section 10.2
Index construction combines dimensionality reduction with UMAP, probabilistic clustering with a Gaussian Mixture Model, and recursive summarization. A probability threshold can let one chunk inform several summaries, making the structure a directed acyclic graph rather than a strict tree; Bayesian Information Criterion selects cluster counts without manual K. section 10.0 section 10.8
Choosing retrieval resolution
For “What is the specific termination fee in the contract?”, retrieval can select a leaf chunk. For “What are the broad conditions under which the contract can be terminated?”, it can select a summary node. Both resolutions remain available through the same retrieval call. section 10.2
The index can be collapsed into one flat searchable index for corpora below about 100,000 chunks, or traversed from the root through winning children for collections reaching millions. Summaries cost LLM calls during indexing, while later queries benefit from the precomputed structure. section 10.8
