The query-key-value framework lets each zone weigh information from other zones. A zone’s query is compared with other zones’ keys; softmax turns the scores into weights, which combine their value vectors into an output. section 9.5
Zone-to-zone attention
For a zone, dot products between its query and other zones’ keys measure relevance. The resulting weights emphasize the values of the most relevant zones. section 9.5
Figure 9.6 The Attention Mechanism — Query, Key, Value.
Use the diagram to follow how queries, keys, and values produce an attention output. section 9.5
