liveBook · Manning

Inner product

As explained in Rearchitecting LLMs

A dot product is a scalar operation between two equal-length vectors that measures their directional alignment.

An inner product compares a Query vector with a Key vector to score their alignment. In attention, those scores are softmax-normalized into weights that determine how much of each Value contributes to the updated token embedding. section 3.1.2

Figure 3.4 Attention mechanism for the token "spring." The Query (Q) from "spring" compares with the Keys (K) of all tokens through dot product operations. The resulting scores, after softmax normalization, determine what fraction of each Value (V) contributes to create the enriched embedding.

See how Query–Key dot products produce scores that softmax turns into Value weights. section 3.1.2

How Rearchitecting LLMs covers it

How deep it goes
Explains alignment scores and their softmax normalization into weights, without deriving the operation mathematically.
Examples
Compares the Query for “spring” with the Key for “mountain” to illustrate a high alignment score.
What you can do after
Understand how alignment scores become attention weights for mixing Value vectors.
How it connects
Connects Query and Key vectors to softmax, attention, and Value weighting.

Also explained in

Compare how 19 books explain inner product →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.