liveBook · Manning

Hierarchical index

As explained in Retrieval Augmented Generation, The Foundational Ideas

A hierarchical index organizes retrieval representations at multiple levels of abstraction, from detailed items to broader summaries.

A hierarchical index organizes retrieval representations as a tree-like structure instead of a flat list of vectors. It addresses semantic flattening, incorrect specificity, and incomplete answers by preserving relationships between detailed passages and broader summaries. section 10.0

A RAPTOR-style index contains multiple retrieval resolutions:

  • Leaf nodes retain original text chunks for specific facts, such as a support phone number or contract fee. section 10.2
  • Summary nodes group semantically related chunks and store LLM-generated abstractions, such as a summary of a termination clause. section 10.2
  • Higher layers recursively summarize lower layers until a root represents the entire corpus. section 10.2

Index construction combines dimensionality reduction with UMAP, probabilistic clustering with a Gaussian Mixture Model, and recursive summarization. A probability threshold can let one chunk inform several summaries, making the structure a directed acyclic graph rather than a strict tree; Bayesian Information Criterion selects cluster counts without manual K. section 10.0 section 10.8

Choosing retrieval resolution

For “What is the specific termination fee in the contract?”, retrieval can select a leaf chunk. For “What are the broad conditions under which the contract can be terminated?”, it can select a summary node. Both resolutions remain available through the same retrieval call. section 10.2

The index can be collapsed into one flat searchable index for corpora below about 100,000 chunks, or traversed from the root through winning children for collections reaching millions. Summaries cost LLM calls during indexing, while later queries benefit from the precomputed structure. section 10.8

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Builds a detailed architecture with leaves, summaries, recursive clustering, and search strategies, while deferring implementation code.
Examples
Uses support-phone lookup, contract termination questions, narrative question answering, and enterprise technical-document search.
What it assumes
Assumes basic retrieval familiarity but teaches UMAP, probabilistic clustering, recursive summarization, and variable-resolution retrieval.
What you can do after
Build it from scratch by clustering chunks, generating recursive summaries, and choosing flat or traversed retrieval for the corpus scale.
How it connects
Connects retrieval-augmented generation, semantic search, UMAP, probabilistic clustering, Gaussian mixture models, and summarization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.