liveBook · Manning

Qkv mechanism

As explained in Rearchitecting LLMs

The QKV mechanism computes query, key, and value vectors so attention can score relationships and combine contextual information.

For each token, attention generates a Query (Q), Key (K), and Value (V). Q searches for relevant information; K represents what a token offers; V carries its semantic content. section 3.1.2

Attention uses these vectors to enrich a token’s embedding:

  • Compare the current token’s Query with each token’s Key using dot products to calculate compatibility scores. section 3.1.2

“Spring” searches for context

In “Fresh water flows from the mountain spring daily,” the Query for “spring” compares with the Keys of other tokens; the resulting scores determine which Values contribute to its context. section 3.1.2

How Rearchitecting LLMs covers it

How deep it goes
Defines Q, K, and V roles and traces dot-product scoring through normalized Value aggregation.
Examples
Uses “spring” seeking context from “mountain” to illustrate Query–Key comparison and Value contributions.
What it assumes
Uses token embeddings and attention heads without establishing their foundations.
What you can do after
Understand how Query–Key scores determine the Value information combined into a token’s context.
How it connects
Connects QKV attention to multi-head attention, grouped-query attention, multi-query attention, and KV caching.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.