A passage can serve more than one theme. In a retrieval system that clusters document chunks before summarizing them, forcing each chunk into one group can leave a relevant summary without essential context. A Gaussian mixture model (GMM) supports soft clustering: it gives each chunk a probability for every cluster, so a chunk can contribute to multiple summaries. section 10.2.1 section 10.3.2
A contract passage with two themes
A clause allowing termination upon vendor insolvency concerns both termination and financial risk. A hard assignment might place it only in the termination cluster, leaving the financial-risk summary incomplete. With a GMM, it can inform both if its probabilities clear the assignment threshold. section 10.3.2
A GMM represents the distribution of embeddings as a weighted sum of Gaussian components. Each component has a weight, a center, and a covariance matrix describing its shape. Fitting the model finds values that explain the observed points; unlike a single-label assignment, its output includes per-cluster probabilities. Covariance also lets components accommodate elongated rather than only spherical clusters. section 10.3.2
In a recursive retrieval pipeline, the probabilities become an indexing decision:
- Split documents into short chunks, embed them, and reduce the embedding dimensions with UMAP before clustering. section 10.4
- Fit a GMM and assign a chunk to each cluster whose probability exceeds a threshold. A loose threshold such as 0.1 favors overlap and recall; raising it toward 0.3–0.5 can help when weakly related chunks make summaries drift. section 10.3.2
- Summarize each cluster, then repeat the process on those summaries. Bayesian Information Criterion can select the number of clusters by balancing fit against complexity. section 10.4 section 10.8
Cluster quality matters more than mathematical sophistication: incoherent clusters can produce poor summaries that propagate upward. Inspect the summaries and adjust the UMAP neighborhood size or GMM threshold when they become garbled. The added clustering and summarization work happens during indexing, buying cheaper queries afterward. section 10.3 section 10.8





