liveBook · Manning

Query

As explained in Rearchitecting LLMs

A query vector is an attention representation used to determine how strongly a position should attend to available keys.

A query (Q) is a vector generated from a token’s embedding to search for contextual information. It is compared with the keys (K) of tokens in the sequence; the resulting compatibility scores determine how much of each token’s value (V) contributes to the query token’s enriched embedding. section 3.1.2

Figure 3.4 Attention mechanism for the token "spring." The Query (Q) from "spring" compares with the Keys (K) of all tokens through dot product operations. The resulting scores, after softmax normalization, determine what fraction of each Value (V) contributes to create the enriched embedding.

The diagram traces how a query is compared with keys and how the resulting scores weight values. section 3.1.2

The token “spring”

In “Fresh water flows from the mountain spring daily,” the query for “spring” searches the other tokens for clues about its context. A high dot product with the key for “mountain” indicates strong alignment; after softmax normalization, the scores determine each value’s contribution. section 3.1.2

How Rearchitecting LLMs covers it

How deep it goes
Works through query-key dot products and softmax-weighted values without detailing the query projection’s implementation.
Examples
Uses the query for “spring” searching for contextual clues, including alignment with the key for “mountain.”
What you can do after
Understand how query-key scores determine the contributions of values to a token’s enriched embedding.
How it connects
Connects queries to keys, values, dot products, softmax, and attention.

Also explained in

Compare how 14 books explain query →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.