liveBook · Manning

Key

As explained in How Large Language Models Work

An attention key is a vector representing the attributes against which an attention query is matched.

An attention key is a vector representing a possible answer that an attention query can match. Each key has a corresponding value vector, which supplies the information returned after matching. Unlike a Python dictionary, attention compares every key by similarity and combines their contributions rather than requiring one exact match. section 3.2.2

Predicting the next token

When predicting the next token, the current token acts as the query and preceding words act as keys. Attention weighs those keys by similarity, helping the model focus on relevant context. section 3.2.2

Figure 3.8 An example of how queries, keys, and values work inside a transformer compared to a Python dictionary. When a Python dictionary matches queries to keys, it needs an exact match to find the value, or it will return nothing. A transformer always returns something based on the most similar matches between queries and keys.
figure

Compare exact dictionary lookup with transformer attention: a dictionary may return nothing without an exact key, while attention returns a weighted result from similar keys. section 3.2.2

How How Large Language Models Work covers it

How deep it goes
Defines keys, relates them to values and queries, and stops short of detailed vector calculations.
Examples
Uses Python dictionary lookup and preceding words in next-token prediction as concrete examples.
What it assumes
Assumes basic vectors and embedding layers while defining query, key, and value roles from the ground up.
What you can do after
Understand how attention uses similarity-weighted keys to select contextual information.
How it connects
Connects keys with attention, queries, values, embedding vectors, and transformer layers.
Cover of How Large Language Models Work
liveBookmanning.com
Read How Large Language Models Work on liveBook Get the book at manning.com

Also explained in

Compare how 8 books explain key →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of How Large Language Models Work. The section links point to the parts of the book it draws on.