liveBook · Manning

Query

As explained in Machines that Think

A query is an attention vector representing the relationship sought by a sequence element when retrieving information from other elements.

A query is a learned vector that represents what a token seeks from its context. In attention, it is compared with other tokens’ keys to assess relevance; the corresponding values carry information that can contribute to the token’s updated representation. section 4.1.3

The query works with two other learned projections:

  • A key provides characteristics used to judge whether its token is relevant to the query. section 4.1.3
  • A value carries information that can be added to the representation when its token receives attention. section 4.1.3

Interpreting “rises”

For “rises” in “The sun rises in the east,” its query can align more strongly with the key for “sun” than with the key for “the.” The comparison produces relevance scores; values, rather than queries, carry the information used to update the representation. section 4.1.3

Figure 4.4 The query for “rises” is compared with the keys of other tokens to produce relevance scores. A stronger match with “sun” gives it a higher score than “the,” indicating that it is more useful for interpreting “rises.”

The diagram shows how the query for “rises” is compared with other tokens’ keys to produce relevance scores. section 4.1.3

Queries also connect encoder and decoder representations in cross-attention. At each decoding step, the decoder’s current representation supplies the query, while the encoder’s output supplies keys and values. This lets the decoder retrieve relevant input information without directly accessing future tokens in its own sequence. section 4.1.7

How Machines that Think covers it

How deep it goes
Traces learned projections and query-key dot-product comparisons, without detailing numerical calculations or projection training.
Examples
Uses “rises” seeking useful context from “sun” and a decoder seeking “east” from the input representations.
What it assumes
Assumes token representations combine meaning and position, while defining queries alongside keys and values.
What you can do after
Understand how queries match keys to identify relevant information in self-attention and cross-attention.
How it connects
Links queries to keys, values, self-attention, cross-attention, encoder-decoder connections, and hidden states.

Also explained in

Compare how 14 books explain query →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.