liveBook · Manning

UMAP

As explained in Retrieval Augmented Generation, The Foundational Ideas

UMAP (Uniform Manifold Approximation and Projection) is a nonlinear dimensionality-reduction technique that constructs a low-dimensional representation while preserving local and some global structure.

UMAP (Uniform Manifold Approximation and Projection) compresses high-dimensional embeddings into a smaller space while preserving nearby relationships and broad cluster topology. It constructs a graph in the original space and optimizes a structurally similar low-dimensional graph, making it useful when distance metrics become uninformative in very large vector spaces. section 10.3.1

In RAPTOR-style retrieval, UMAP reduces text-embedding vectors before Gaussian Mixture Model clustering. This addresses semantic flattening: clusters can organize chunks into themes that later support recursive summaries rather than leaving retrieval as a flat list of vectors. section 10.0 section 10.3.1

The main control is n_neighbors, which changes the balance between local detail and broad structure:

  • Small values such as 2–10 emphasize local neighborhoods and produce tight, potentially fragmented groups suited to specific subtopics. section 10.3.1
  • Large values such as 30–100 emphasize global connections and suit higher-level grouping of broad themes. section 10.3.1
  • Reducing to roughly 2–10 components is used to improve the effectiveness of subsequent clustering. section 10.3.1
  • Clamp n_neighbors to n_samples - 2 as the dataset shrinks during recursive tree construction, or UMAP can raise a ValueError. section 10.4.2

Reducing embedding vectors before clustering

A pipeline can embed document leaves with text-embedding-3-small, reduce the resulting 1536-dimensional vectors with UMAP, and pass the reduced representation to soft GMM clustering. The recursive process then summarizes each cluster and repeats until a root summary remains. section 10.4

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains graph construction, local/global preservation, parameter ranges, and a recursive-build guard, but omits mathematical proofs.
Examples
Uses 1536-dimensional text embeddings, 2–10 output dimensions, recursive document chunks, and adjustable neighborhood sizes as implementation examples.
What it assumes
Assumes embeddings and basic clustering concepts while explaining dimensionality problems and parameter effects without requiring manifold-learning proofs.
What you can do after
Apply it via umap-learn to reduce embedding vectors and configure neighborhood size for local or broad thematic clustering.
How it connects
Connects dimensionality reduction with manifold learning, text embeddings, Gaussian Mixture Models, clustering, and hierarchical retrieval.

Also explained in

Compare how 5 books explain UMAP →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.