liveBook · Manning

Soft clustering

As explained in Retrieval Augmented Generation, The Foundational Ideas

Soft clustering assigns an item degrees of membership in multiple clusters instead of assigning it to exactly one cluster.

A passage can belong to more than one theme. In a flat retrieval index, short chunks preserve specific facts but lose the relationships needed to answer broader questions. Soft clustering helps rebuild those relationships: it lets a chunk contribute to several groups rather than forcing it across one classification boundary. section 10.0 section 10.2.1

In RAPTOR, short chunks are embedded, clustered, and summarized. Each summary becomes text for the next round of embedding, clustering, and summarization. Unlike hard clustering such as K-Means, a Gaussian Mixture Model (GMM) gives each chunk a probability for every cluster. A chunk joins each cluster whose probability clears a chosen threshold, so one passage can inform multiple parent summaries. The resulting structure is a directed acyclic graph, not a strict tree. section 10.2.1 section 10.3.2 section 10.4.2

One passage, two contexts

An insolvency paragraph can join both a termination cluster and a financial-risk cluster. Each cluster’s summary then retains the passage in its own context. This overlap reduces the chance that a relevant fact is hidden behind an arbitrary boundary. section 10.3.2

Build and tune the clusters at indexing time:

  • Project embeddings with UMAP before clustering. As recursive layers shrink, clamp n_neighbors to n_samples − 2 to avoid a build-time error. section 10.4.2
  • Sweep candidate GMM cluster counts and select the lowest Bayesian Information Criterion (BIC), rather than fixing the number of clusters in advance. Use a fixed seed for reproducible runs and a covariance regularization floor to prevent the tree from failing to narrow. section 10.4.2
  • Start with a membership threshold around 0.1 for generous overlap. Raise it toward 0.3–0.5 if weakly related chunks make summaries drift; lower it if upper layers become sparse. If no probability clears the threshold, assign the chunk to its highest-probability cluster so it remains reachable. section 10.3.2 section 10.4.2
Listing 10.3 Running the clustering engine over a level of the tree.
engine = ClusteringEngine(threshold=0.1, seed=42)
assignments = engine.perform_clustering(embeddings)  #A
 
for chunk_idx, cluster_ids in enumerate(assignments[:6]):
    print(f"chunk {chunk_idx} -> clusters {cluster_ids}")
 

The clustering output lists cluster IDs for each chunk. A chunk with two IDs is a bridge between two summaries at the next level. section 10.4.2

The extra clustering and LLM summarization work happens during indexing. Indexing summaries alongside raw chunks lets a broad question match an abstraction while a factual question can still match a leaf. Evaluate the threshold against both missing context and summaries that blend unrelated material; greater overlap is not automatically better. section 10.8 section 10.3.2

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains GMM membership probabilities, thresholding, cluster-count selection, and implementation safeguards; keeps the mathematical treatment light.
Examples
Uses an insolvency passage assigned to termination and financial-risk summaries, plus Python output showing a chunk assigned two cluster IDs.
What it assumes
Assumes basic RAG and chunk embeddings; introduces the probabilities needed to follow clustering before turning to implementation decisions.
What you can do after
Build it from scratch by selecting a GMM cluster count, assigning chunks by probability threshold, and checking overlap against summary drift.
How it connects
Connects soft clustering to embeddings, UMAP, Gaussian Mixture Models, BIC, recursive summarization, and retrieval-augmented generation.

Also explained in

Compare how 3 books explain soft clustering →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.