Value
In attention, a value is a projected vector representation of content that is combined with other values according to attention weights to produce an output.
Timeless Algorithms, The Foundational Ideas
A projected representation of information that is combined with other values according to attention weights.
- How it is used
- Uses value vectors to form contextual token representations through attention.
- How deep it goes
- Works through a two-dimensional weighted combination, while noting that real transformers use larger representations and additional components.
- Examples
- Calculates a representation for “it” from weighted trophy and box vectors.
- What you can do after
- Evaluate/understand how attention weights change the contribution of each value to a contextual representation.
Read how Timeless Algorithms, The Foundational Ideas explains value →
Transformers in Action
In attention, a value is a vector representation containing the content aggregated according to attention weights.
- How it is used
- Uses values as the content aggregated for each query in transformer attention.
- How deep it goes
- Explains weighted aggregation and value projection, then demonstrates simplified scaled dot-product and multihead attention implementations.
- Examples
- Uses a sentence about a movie and a four-embedding numerical implementation to illustrate weighted content retrieval.
- What you can do after
- Builds a simplified scaled dot-product attention calculation that projects and aggregates values.
Build a DeepSeek Model (From Scratch)
A value is an attention representation containing the information contributed by a token when its key is selected.
- How it is used
- Explains how token information enters context vectors and why value reuse matters during autoregressive inference.
- How deep it goes
- Derives value projection and weighted context with matrix shapes, then traces caching and alternative value representations.
- Examples
- Uses four-token and five-token attention calculations, plus a four-head grouped-query arrangement.
- What you can do after
- Evaluate and understand how values contribute to context and which computations caching avoids.
Read how Build a DeepSeek Model (From Scratch) explains value →
Build a Text-to-Image Generator (from Scratch)
A value is an attention vector whose information is combined according to attention weights.
- How it is used
- Carries the information that attention weights select and combine in transformer, vision-transformer, and diffusion U-Net computations.
- How deep it goes
- Traces value projection, scaled similarity, softmax weighting, head splitting, and output recombination through implementation-level detail.
- Examples
- Uses German-English token translation, four-head image attention, and four-head attention over diffusion U-Net feature maps.
- What you can do after
- Builds value projections and weighted attention outputs with PyTorch linear layers, softmax, matrix multiplication, and tensor reshaping.
Read how Build a Text-to-Image Generator (from Scratch) explains value →
Sutskever's List
- How it is used
- Uses projected representations as the information attention combines to form outputs during Transformer cross-attention.
- How deep it goes
- Defines values as projected representations combined by attention weights, but supplies no derivation or implementation details.
- Examples
- Illustrates query, key, and value interaction with a Transformer encoder-decoder schematic.
- What you can do after
- Understand how cross-attention uses values to contribute encoded information to a context vector guiding token generation.
CUDA for LLMs
An attention value is a vector combined with other values according to attention weights.
- How it is used
- Supplies the vectors that attention weights combine to gather information across sequence positions.
- How deep it goes
- Shows value projection and weighted aggregation in a PyTorch head, but does not derive the value representation.
- Examples
- Uses a PyTorch attention head whose softmax weights multiply projected values.
- What you can do after
- Apply via PyTorch by projecting inputs to values and multiplying them by softmax attention weights.
Learn Generative AI with PyTorch
- How it is used
- Combines projected content vectors according to attention weights to produce an output in self-attention.
- How deep it goes
- Traces value projection and weighted combination in scaled dot-product attention, without detailing the layer implementation.
- Examples
- Uses library search results as an analogy for values selected through query-key similarity.
- What you can do after
- Build the attention mechanism from scratch using query, key, and value vectors to calculate scaled dot-product attention.
Build AI Drug Discovery Pipelines
A value vector is the attention representation combined with weights to form an attended output.
- How it is used
- Provides the information retrieved from encoder states during cross-attention.
- How deep it goes
- Explains values through their weighted-sum role in producing context vectors, without descending into parameter-level implementation.
- Examples
- Uses encoder hidden states and decoder prediction steps as the concrete artifacts for explaining value aggregation.
- What you can do after
- Evaluate/understand how attention weights select and combine encoder information for each decoder output.
Applied Reinforcement Learning
A value is an attention vector containing the information contributed by an input element after attention weights are computed.
- How it is used
- Combines input-element information according to attention weights to form each element’s attention output.
- How deep it goes
- Defines values as vectors and explains their weighted-sum contribution to attention outputs.
- Examples
- Uses building zones as input elements and considers scaling attention from 8 zones to 80.
- What you can do after
- Understand how attention weights select and combine information from relevant neighboring zones.
Deep Learning with PyTorch, Second Edition
The transformed representation vectors that are combined using attention weights to produce the output for each position.
- How it is used
- Uses transformed content vectors to integrate context into each sequence position’s attention output.
- How deep it goes
- Explains linear value projection and weighted summation, while treating query-key weight calculation as the surrounding attention mechanism.
- Examples
- Demonstrates character embeddings, next-character prediction, and a PyTorch calculation using F.linear, softmax, and matrix multiplication.
- What you can do after
- Apply via PyTorch by projecting inputs into values and combining them with attention weights.
Read how Deep Learning with PyTorch, Second Edition explains value →
Build a Large Language Model (From Scratch)
The attention content vector combined according to attention weights to form the context output.
- How it is used
- Carries transformed token content into the weighted sum that produces each self-attention context vector.
- How deep it goes
- Explains W_value projection, tensor shapes, and weighted combination, stopping at a single-query example rather than multi-head construction.
- Examples
- Uses the six-word sentence “Your journey starts with one step,” second-input query x_2, and PyTorch tensors shaped [6, 2].
- What you can do after
- Build it from scratch in PyTorch by projecting inputs with W_value and combining values with attention weights.
Read how Build a Large Language Model (From Scratch) explains value →
Deep Learning with Python, Third Edition
- How it is used
- Uses projected content vectors as the inputs combined by attention scores to produce an output.
- How deep it goes
- Defines values through the weighted-sum operation and contrasts them with keys, without deriving the projections.
- Examples
- Uses photo retrieval to compare query terms, photo tags, and returned images.
- What you can do after
- Understand how attention weights combine value vectors into an output.
Read how Deep Learning with Python, Third Edition explains value →
Understanding AI
A value vector is an attention vector that carries the information contributed by a token after attention weights are applied.
- How it is used
- Carries token information into the context-aware representation produced by attention.
- How deep it goes
- Explains value vectors through roles, dot-product scores, softmax weights, and weighted summation without deriving the mathematics.
- Examples
- Uses the sentence about an unbelievable performance and pronoun “it,” with illustrative attention weights.
- What you can do after
- Understand how attention combines token information into an enriched representation.
Grokking AI Algorithms, Second Edition
A value vector is the vector representation carrying the content contributed by a token in an attention computation.
- How it is used
- Uses token content to build context-rich representations during self-attention.
- How deep it goes
- Explains projection, weighting, elementwise scaling, and summation, stopping short of deeper training mechanics.
- Examples
- Works through token 0’s four-dimensional Value vector, weighted results, and final context vector.
- What you can do after
- Apply it via Python by projecting embeddings into V and multiplying V by attention weights.
Read how Grokking AI Algorithms, Second Edition explains value →
Rearchitecting LLMs
A value vector is the attention representation combined according to attention weights to produce an attended output.
- How it is used
- Describes the attention representation combined using attention weights to produce an attended output.
- How deep it goes
- Defines weighted value aggregation and identifies GQA projection sharing, but does not detail the computation.
- Examples
- Uses Llama-3.2-1B projections, where v_proj maps 2048 input features to 512 output features.
- What you can do after
- Understand how GQA shares value projections across heads within a group.
Also covered in
What to know first
- transformer architectureA transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.
- softmaxSoftmax is a function that exponentiates each value in a score vector and normalizes the results to produce positive values summing to one, forming a probability distribution.
- embeddingAn embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
- recurrent neural networkA recurrent neural network is a neural network that processes ordered inputs while carrying and updating an internal state across successive steps.
Related concepts
- attentionused by · Attention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
- attention weightused by · An attention weight is a coefficient indicating how strongly an associated input or value contributes to an attention-based output or representation.
- key-value cachepart of · A key-value cache stores attention keys and values computed for earlier tokens so they can be reused during subsequent autoregressive generation.
- context matrixproduces · A context matrix contains attention outputs for all sequence positions, with each row formed by weighting value vectors.
- context vectorproduces · A context vector is a numerical vector that summarizes information from relevant input context for a model’s prediction or subsequent processing.
- grouped-query attentionused by · Grouped-query attention is an attention mechanism in which multiple query heads share key and value heads or projections.
Questions about value
What is value?
In attention, a value is a projected vector representation of content that is combined with other values according to attention weights to produce an output.
Which Manning books explain value?
Timeless Algorithms, The Foundational Ideas, Transformers in Action, Build a DeepSeek Model (From Scratch), Build a Text-to-Image Generator (from Scratch), Sutskever's List, CUDA for LLMs, Learn Generative AI with PyTorch, Build AI Drug Discovery Pipelines and 7 more.
What should I know before learning value?
Manning books that teach value usually introduce transformer architecture, softmax, embedding and recurrent neural network first.
What is value part of?
Value is part of key-value cache. A key-value cache stores attention keys and values computed for earlier tokens so they can be reused during subsequent autoregressive generation.
What is value also called?
Value is also called V.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.














