liveBook · Manning

Inner product

As explained in Retrieval Augmented Generation, The Foundational Ideas

An inner product is an operation that maps two vectors to a scalar expressing their algebraic alignment.

An inner product combines two vectors into a scalar that expresses their algebraic alignment. In semantic search, it compares a query vector q with database vectors xᵢ; scanning every vector costs O(n × d), which becomes impractical at enterprise scale. section 2.2.1

The production tradeoff is straightforward:

  • A small catalog can use exact dot-product scans; a 6,500-movie catalog with 128-dimensional embeddings required fewer than one million floating-point operations per query. section 2.2.1
  • Global catalogs and web-scale document collections require approximate nearest-neighbor algorithms instead of real-time brute-force comparison. section 2.2.1
  • Normalizing vectors makes their dot product equal cosine similarity, emphasizing direction while discarding magnitude. section 2.2.1
Figure 2.2 What cosine similarity measures and discards. Top left: the query and doc A sit at angle θ, and doc B, twice as long and pointing the same way, scores identically, because normalizing removes magnitude before the dot product. The other three panels draw the values listed above: identical direction scores 1.0, right angles score 0.0, and opposite directions score −1.0, which text embeddings almost never produce.

Use the figure to see why equally directed vectors receive the same cosine score after normalization, while right angles score 0.0 and identical directions score 1.0. section 2.2.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines brute-force cost and cosine interpretation, but stops short of deriving ANN algorithms or implementation details.
Examples
Uses Netflix’s 6,500-movie catalog, Amazon’s 200-million-item catalog, and normalized query-document vectors.
What you can do after
Evaluate/understand when brute-force vector comparison becomes impractical and why cosine similarity or ANN is used.
How it connects
Connects inner products to vector embeddings, cosine similarity, semantic search, and approximate nearest-neighbor algorithms.

Also explained in

Compare how 19 books explain inner product →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.