liveBook · Manning

Key

As explained in Build AI Drug Discovery Pipelines

A key vector is an attention representation compared with a query to determine the relevance of an associated value.

A key vector is an encoder hidden state that a decoder compares with its current query to estimate relevance. The resulting attention weights determine how strongly the associated value contributes to the context vector. In protein structure prediction, keys let each output position retrieve information from nearby or distant input positions instead of relying on one final bottleneck vector. section 12.3.1

Within cross-attention, the three representations have distinct jobs:

  • Query: represents the decoder's current focus and information need. section 12.3.1
  • Key: represents each encoder position available for comparison. section 12.3.1
  • Value: supplies the encoder information combined into the context vector after weighting. section 12.3.1
Figure 12.8 The mechanics of cross-attention. A decoder state acts as a query (qj) and compares itself against all encoder states (keys, K). These comparisons generate attention weights (α), which are used to compute a weighted sum of the encoder states (values, V), creating a context vector (cj) specific to the current prediction step.

Trace how a decoder query compares with encoder keys, produces attention weights, and combines values into a prediction-specific context vector. section 12.3.1

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Defines keys operationally through query comparisons and attention weights, but does not derive their mathematical scoring function.
Examples
Uses protein structure prediction, output position j, and nearby or distant three-dimensional interactions as attention examples.
What it assumes
Assumes encoder hidden states and decoder steps, while defining keys alongside queries, values, and context vectors.
What you can do after
Evaluate/understand how decoder states select relevant encoder positions through query-key comparisons.
How it connects
Connects keys with cross-attention, queries, values, encoder hidden states, attention weights, and context vectors.

Also explained in

Compare how 15 books explain key →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.