liveBook · Manning

Value

As explained in Understanding AI

A value vector is an attention vector that carries the information contributed by a token after attention weights are applied.

A value vector is the information a token offers when attention builds a context-aware representation. Attention first decides how relevant each token is, then uses those weights to combine their values. section 2.1.3

The attention process works in a short sequence:

  • Each token supplies a query, a key, and a value: what it seeks, what it offers, and what information it carries. section 2.1.3
  • The query compares with other tokens' keys using dot products, producing relevance scores. section 2.1.3
  • Softmax turns the scores into weights, and the model multiplies each value by its weight before summing the results. section 2.1.3

Resolving “it”

In “The performance was unbelievable—the crowd couldn't believe it,” the query for “it” gives “performance” the strongest weight, “crowd” a smaller one, and “couldn't” almost none. The weighted value sum enriches “it” with mostly information from “performance.” section 2.1.3

Figure 2.3 When processing "it," the model assigns an attention weight to every other token in the sentence. "Performance" receives the highest weight by far (0.68 out of 1.0)which is how the model infers what "it" refers to.

Use the attention figure to see how weights assigned to surrounding tokens determine which value information contributes most strongly. section 2.1.3

How Understanding AI covers it

How deep it goes
Explains value vectors through roles, dot-product scores, softmax weights, and weighted summation without deriving the mathematics.
Examples
Uses the sentence about an unbelievable performance and pronoun “it,” with illustrative attention weights.
What it assumes
Assumes token representations and introduces query, key, and value roles while explaining their interaction from the basics.
What you can do after
Understand how attention combines token information into an enriched representation.
How it connects
Links value vectors to attention, queries, keys, dot products, softmax, token representations, and next-token prediction.

Also explained in

Compare how 15 books explain value →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Understanding AI. The section links point to the parts of the book it draws on.