A key vector is an attention representation compared with a query to measure compatibility. Attention uses those scores to select and combine corresponding value vectors. section 15.3.1
A database lookup analogy
Imagine searching a photo database: the query is the search term, keys are photo tags matched against it, and values are the retrieved photos. Attention performs a comparable lookup over source vectors. section 15.3.1
In parameterized attention, learned Dense layers project query, key, and value inputs into useful spaces. Dot products between each query and the keys produce scores; softmax turns them into weights for combining values. With causal attention, a position compares its query with previous key vectors while a causal mask blocks future positions. section 15.3.1 section 16.2.3
Key vectors also support efficient Transformer generation:
- Past key vectors remain unchanged because past input is fixed and causal attention prevents looking ahead. section 16.2.3
- Caching key and value vectors at every Transformer layer lets generation process one new position at a time, potentially producing a thousandfold speedup for long sequences. section 16.2.3
