Value
In attention, a value is a vector representation of input content that is combined with other values according to attention weights to produce an output.
Transformers in Action
In attention, a value is a vector representation containing the content aggregated according to attention weights.
- How it is used
- Supplies the content combined by attention weights to produce contextual output vectors.
- How deep it goes
- Explains the value role intuitively and implements its projection and weighted aggregation in simplified attention code.
- Examples
- Uses a sentence about a movie and a NumPy scaled dot-product attention listing.
- What you can do after
- Build it from scratch in NumPy by projecting embeddings into V and multiplying V by attention scores.
Build a Text-to-Image Generator (from Scratch)
A value is an attention vector whose information is combined according to attention weights.
- How it is used
- Carries the information that attention weights select and combine in transformer, vision-transformer, and diffusion U-Net computations.
- How deep it goes
- Traces value projection, scaled similarity, softmax weighting, head splitting, and output recombination through implementation-level detail.
- Examples
- Uses German-English token translation, four-head image attention, and four-head attention over diffusion U-Net feature maps.
- What you can do after
- Builds value projections and weighted attention outputs with PyTorch linear layers, softmax, matrix multiplication, and tensor reshaping.
Read how Build a Text-to-Image Generator (from Scratch) explains value →
Sutskever's List
- How it is used
- Uses projected representations as the information attention combines to form outputs during Transformer cross-attention.
- How deep it goes
- Defines values as projected representations combined by attention weights, but supplies no derivation or implementation details.
- Examples
- Illustrates query, key, and value interaction with a Transformer encoder-decoder schematic.
- What you can do after
- Understand how cross-attention uses values to contribute encoded information to a context vector guiding token generation.
Learn Generative AI with PyTorch
- How it is used
- Uses projected values to form the output of self-attention in a Transformer for language translation.
- How deep it goes
- Traces values from input embedding through the W_V projection to their weighted combination in the attention output.
- Examples
- Illustrates values with recommended books in a library-search analogy and an English-to-French translation task.
- What you can do after
- Build self-attention from scratch by projecting inputs into values and combining them with attention weights.
Deep Learning with PyTorch, Second Edition
The transformed representation vectors that are combined using attention weights to produce the output for each position.
- How it is used
- Uses transformed content vectors to integrate context into each sequence position’s attention output.
- How deep it goes
- Explains linear value projection and weighted summation, while treating query-key weight calculation as the surrounding attention mechanism.
- Examples
- Demonstrates character embeddings, next-character prediction, and a PyTorch calculation using F.linear, softmax, and matrix multiplication.
- What you can do after
- Apply via PyTorch by projecting inputs into values and combining them with attention weights.
Read how Deep Learning with PyTorch, Second Edition explains value →
Build a Large Language Model (From Scratch)
The attention content vector combined according to attention weights to form the context output.
- How it is used
- Carries transformed token content into the weighted sum that produces each self-attention context vector.
- How deep it goes
- Explains W_value projection, tensor shapes, and weighted combination, stopping at a single-query example rather than multi-head construction.
- Examples
- Uses the six-word sentence “Your journey starts with one step,” second-input query x_2, and PyTorch tensors shaped [6, 2].
- What you can do after
- Build it from scratch in PyTorch by projecting inputs with W_value and combining values with attention weights.
Read how Build a Large Language Model (From Scratch) explains value →
Deep Learning with Python, Third Edition
- How it is used
- Combines source-content vectors according to attention scores to form the attention output.
- How deep it goes
- Explains weighted summation and separate value projections with parameterized attention code.
- Examples
- Uses photo retrieval as an analogy and projected source vectors in parameterized attention code.
- What you can do after
- Understand how attention scores weight value vectors to produce an output.
Read how Deep Learning with Python, Third Edition explains value →
Grokking AI Algorithms, Second Edition
A value vector is the vector representation carrying the content contributed by a token in an attention computation.
- How it is used
- Uses token content to build context-rich representations during self-attention.
- How deep it goes
- Explains projection, weighting, elementwise scaling, and summation, stopping short of deeper training mechanics.
- Examples
- Works through token 0’s four-dimensional Value vector, weighted results, and final context vector.
- What you can do after
- Apply it via Python by projecting embeddings into V and multiplying V by attention weights.
Read how Grokking AI Algorithms, Second Edition explains value →
Related concepts
- key-value cachepart of · A key-value cache is a data structure that stores previously computed transformer attention keys and values for reuse during subsequent token generation.
- attentionused by · Attention is a neural-network mechanism that computes input-dependent weights to combine information according to its relevance to a computation.
- attention weightused by · An attention weight is a computed coefficient, often normalized, that determines how strongly an input element or value contributes to an attention-based representation.
- grouped-query attentionused by · Grouped-query attention partitions query heads into groups whose members share key and value projections, reducing attention computation and memory use.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.