liveBook · Manning

Key

As explained in Rearchitecting LLMs

A key vector is an attention representation matched against queries to determine attention weights.

A key vector represents a token for comparison against a query. Dot-product scores are softmax-normalized into weights that determine how much each token’s value contributes to the query token’s enriched embedding. section 3.1.2

In grouped-query attention, query heads within a group share key and value vectors rather than each using an independent set. section 3.2.1

Disambiguating “spring”

A query for “spring” compares against the keys for all tokens. The resulting weights let context such as “water” and “mountain,” or “flowers” and “season,” shape its representation. section 3.1.2

Figure 3.4 Attention mechanism for the token "spring." The Query (Q) from "spring" compares with the Keys (K) of all tokens through dot product operations. The resulting scores, after softmax normalization, determine what fraction of each Value (V) contributes to create the enriched embedding.

Shows a query comparing with all token keys and using the normalized scores to weight their values. section 3.1.2

How Rearchitecting LLMs covers it

How deep it goes
Traces query-key dot products through softmax weighting of values and contrasts independent with shared key-value sets.
Examples
Uses “spring” in water-and-mountain and flowers-and-season contexts to illustrate contextual weighting.
What you can do after
Understand how query-key scores allocate value contributions and how grouped-query attention shares keys across query heads.
How it connects
Connects keys to queries, values, self-attention, multi-head attention, and grouped-query attention.

Also explained in

Compare how 15 books explain key →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.