liveBook · Manning

Key

As explained in Understanding AI

A key vector is an attention vector that represents the information a token offers for matching against queries.

When a model processes one token, its key answers a simple question: “What information do I have to offer?” Other tokens compare their queries against that key to decide whether it is relevant. section 2.1.3

The model compares a query with every token’s key using a dot product. Higher matches receive larger softmax weights, allowing the matching token’s value to contribute more to the updated representation. section 2.1.3

Resolving “it”

In “The performance was unbelievable—the crowd couldn’t believe it,” the query for “it” matches “performance” strongly, “crowd” weakly, and “couldn’t” hardly at all. The key for “performance” therefore helps make that token the most relevant source of information. section 2.1.3

Figure 2.3 When processing "it," the model assigns an attention weight to every other token in the sentence. "Performance" receives the highest weight by far (0.68 out of 1.0)which is how the model infers what "it" refers to.

Use the attention-weight figure to see how one token’s query assigns different relevance weights to the keys of the surrounding tokens. section 2.1.3

How Understanding AI covers it

How deep it goes
Explains key-query matching through dot products, softmax weights, and value aggregation.
Examples
Uses pronoun resolution in “The performance was unbelievable—the crowd couldn’t believe it” and an attention-weight figure.
What it assumes
Introduces keys alongside queries and values while assuming only a basic understanding of token representations.
What you can do after
Evaluate/understand how attention identifies relevant tokens by comparing queries with keys.
How it connects
Connects keys with queries, values, attention, embeddings, dot products, softmax, and token representations.

Also explained in

Compare how 15 books explain key →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Understanding AI. The section links point to the parts of the book it draws on.