liveBook · Manning

HNSW

As explained in Generative AI in Action

HNSW is a graph-based approximate nearest-neighbor indexing algorithm that organizes vectors in navigable hierarchical layers.

HNSW (hierarchical navigable small world) is a graph-based approximate nearest-neighbor algorithm for organizing embedding vectors into searchable layers. It accelerates similarity search when retrieval speed and computational overhead matter. section 7.5.1 section 8.3

A vector index arranges embeddings so a query can find nearby vectors without comparing against every stored vector. HNSW uses a graph structure, while FLAT performs brute-force comparison and is described as more accurate but slower and more computationally intensive. section 7.5.1 section 8.3

Configure HNSW by choosing the surrounding vector-search settings:

  • Store vectors as FLOAT32 and set the dimension to match the embedding model. section 8.3
  • Select a distance metric such as EUCLIDEAN, DOTPRODUCT, or COSINE. section 8.3
  • Balance search speed, accuracy, and memory consumption when choosing the index method. section 7.5.1

Redis blog-post index

A Redis Search schema stores blog-post metadata alongside an HNSW VectorField: FLOAT32 embeddings, 1536 dimensions, and COSINE distance. This supports similarity search over embedded content while retaining URL, title, description, publish date, and content fields. section 8.3

Listing 8.2 Redis index schema
SCHEMA = [
    TagField("url"),
    TextField("title"), 
    TextField("description"),
    TextField("publish_date"),
    TextField("content"),
    VectorField("embedding", "HNSW",
                {"TYPE": "FLOAT32",
                 "DIM": 1536,
                 "DISTANCE_METRIC": "COSINE"}),
   ]

Inspect the Redis index schema showing HNSW configuration alongside metadata fields. section 8.3

Table 8.2 HNSW distance metric options
HNSW distance metric
Description
EUCLIDEAN
The straight-line distance between two points in Euclidean space. It’s a good choice when all dimensions are similar (e.g., all distances measured in meters).
DOTPRODUCT
Calculates the dot product between two vectors. The dot product is the sum of the products of the corresponding entries of the two sequence numbers.
COSINE
Calculates the cosine of the angle between two vectors. Regardless of their magnitude, it measures how similar the vectors are. This is often used in text analysis, where the direction of the vector (the angle) is more important than the length of the vector.

Compare the available HNSW distance metrics and their intended similarity interpretations. section 8.3

How Generative AI in Action covers it

How deep it goes
Defines graph organization, contrasts HNSW with brute-force FLAT, and specifies metrics and schema settings without deriving graph internals.
Examples
Uses a Redis blog-post index with FLOAT32 embeddings, 1536 dimensions, and COSINE distance.
What it assumes
Assumes vector embeddings and similarity metrics, while defining HNSW and contrasting exact and approximate search.
What you can do after
Operate/configure Redis vector search by selecting HNSW and setting vector type, dimensions, and distance metric.
How it connects
Connects vector indexes with embeddings, approximate nearest-neighbor search, distance metrics, and Redis vector fields.
Cover of Generative AI in Action
liveBookmanning.com
Read Generative AI in Action on liveBook Get the book at manning.com

Also explained in

Compare how 7 books explain HNSW →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Generative AI in Action. The section links point to the parts of the book it draws on.