A flat RAG index can find a sentence containing an armistice date yet struggle to explain how a political figure’s foreign policy evolved. The second question needs connections across passages, not just a nearby chunk. RAPTOR addresses this semantic flattening by indexing both original chunks and summaries at several levels of abstraction. section 10.0 section 10.2
RAPTOR builds its hierarchy from the bottom up. Short text chunks form leaves. Related chunks are clustered, and an LLM writes a summary for each cluster. Those summaries are embedded, clustered, and summarized again until the process reaches a root or a chosen depth. A leaf can contribute to multiple summaries through soft clustering, so the resulting structure is not always a strict tree. section 10.2.1 section 10.8
A practical indexing pipeline separates five jobs:
- Split text into roughly 100-token leaves and embed them, preserving fine-grained facts for retrieval. section 10.2.1
- Reduce embedding dimensions with UMAP, then use a Gaussian Mixture Model for soft cluster assignments. Bayesian Information Criterion can select the number of clusters. section 10.4 section 10.8
- Summarize each cluster with an LLM and feed the summaries into the next pass. This work happens during indexing, rather than for every query. section 10.2.1 section 10.8
At query time, one option is to place leaves and summaries in the same vector index. Similarity search can then return a leaf for a precise fact or a summary for a broad question; node-level metadata allows filtering by abstraction level. For fewer than about 100,000 chunks, this collapsed index is the suggested default. Root-to-child traversal is slower but remains an option for collections in the millions. section 10.5.1 section 10.8
Two questions, one index
A question about a story’s moral arc retrieves a summary; a question about what became a carriage retrieves leaf chunks. The retrieval call need not classify the question first: the indexed levels supply different kinds of evidence. section 10.5.1
Listing 10.7 Collapsed Tree Indexing.
from langchain_core.documents import Document
from langchain_community.vectorstores import Chroma
def index_collapsed_tree(nodes: Dict[str, RaptorNode],
persist_directory: str) -> Chroma:
documents = []
for node_id, node in nodes.items():
metadata = {
"id": node.id,
"level": node.level,
"is_leaf": node.is_leaf(),
"children_count": len(node.children) #A
}
doc = Document(
page_content=node.text,
metadata=metadata
)
documents.append(doc)
vectorstore = Chroma.from_documents(
documents=documents,
embedding=get_embeddings(),
collection_name="raptor_collapsed",
persist_directory=persist_directory
)
return vectorstore
The indexing code places every node in one Chroma collection and records its level, leaf status, and number of children for later filtering. section 10.5.1
The extra summaries have an indexing cost, and clustering by similarity can cross boundaries it should respect. Similar-sounding defect logs from different products, for example, may produce a misleading shared summary. Partition the corpus before building when product or tenant boundaries are mandatory; otherwise, carry provenance and filter retrieval. Because clustering depends on the corpus, adding documents may require rebuilding rather than simply inserting a new leaf. section 10.5.3 section 10.2.1
