liveBook · Manning

Value

Also called V

In attention, a value is a projected vector representation of content that is combined with other values according to attention weights to produce an output.

How 15 Manning books explain it

Timeless Algorithms, The Foundational Ideas

A projected representation of information that is combined with other values according to attention weights.

How it is used
Uses value vectors to form contextual token representations through attention.
How deep it goes
Works through a two-dimensional weighted combination, while noting that real transformers use larger representations and additional components.
Examples
Calculates a representation for “it” from weighted trophy and box vectors.
What you can do after
Evaluate/understand how attention weights change the contribution of each value to a contextual representation.

Read how Timeless Algorithms, The Foundational Ideas explains value →

Transformers in Action

In attention, a value is a vector representation containing the content aggregated according to attention weights.

How it is used
Uses values as the content aggregated for each query in transformer attention.
How deep it goes
Explains weighted aggregation and value projection, then demonstrates simplified scaled dot-product and multihead attention implementations.
Examples
Uses a sentence about a movie and a four-embedding numerical implementation to illustrate weighted content retrieval.
What you can do after
Builds a simplified scaled dot-product attention calculation that projects and aggregates values.

Read how Transformers in Action explains value →

Build a DeepSeek Model (From Scratch)

A value is an attention representation containing the information contributed by a token when its key is selected.

How it is used
Explains how token information enters context vectors and why value reuse matters during autoregressive inference.
How deep it goes
Derives value projection and weighted context with matrix shapes, then traces caching and alternative value representations.
Examples
Uses four-token and five-token attention calculations, plus a four-head grouped-query arrangement.
What you can do after
Evaluate and understand how values contribute to context and which computations caching avoids.

Read how Build a DeepSeek Model (From Scratch) explains value →

Build a Text-to-Image Generator (from Scratch)

A value is an attention vector whose information is combined according to attention weights.

How it is used
Carries the information that attention weights select and combine in transformer, vision-transformer, and diffusion U-Net computations.
How deep it goes
Traces value projection, scaled similarity, softmax weighting, head splitting, and output recombination through implementation-level detail.
Examples
Uses German-English token translation, four-head image attention, and four-head attention over diffusion U-Net feature maps.
What you can do after
Builds value projections and weighted attention outputs with PyTorch linear layers, softmax, matrix multiplication, and tensor reshaping.

Read how Build a Text-to-Image Generator (from Scratch) explains value →

Sutskever's List

How it is used
Uses projected representations as the information attention combines to form outputs during Transformer cross-attention.
How deep it goes
Defines values as projected representations combined by attention weights, but supplies no derivation or implementation details.
Examples
Illustrates query, key, and value interaction with a Transformer encoder-decoder schematic.
What you can do after
Understand how cross-attention uses values to contribute encoded information to a context vector guiding token generation.

Read how Sutskever's List explains value →

CUDA for LLMs

An attention value is a vector combined with other values according to attention weights.

How it is used
Supplies the vectors that attention weights combine to gather information across sequence positions.
How deep it goes
Shows value projection and weighted aggregation in a PyTorch head, but does not derive the value representation.
Examples
Uses a PyTorch attention head whose softmax weights multiply projected values.
What you can do after
Apply via PyTorch by projecting inputs to values and multiplying them by softmax attention weights.

Read how CUDA for LLMs explains value →

Learn Generative AI with PyTorch

How it is used
Combines projected content vectors according to attention weights to produce an output in self-attention.
How deep it goes
Traces value projection and weighted combination in scaled dot-product attention, without detailing the layer implementation.
Examples
Uses library search results as an analogy for values selected through query-key similarity.
What you can do after
Build the attention mechanism from scratch using query, key, and value vectors to calculate scaled dot-product attention.

Read how Learn Generative AI with PyTorch explains value →

Build AI Drug Discovery Pipelines

A value vector is the attention representation combined with weights to form an attended output.

How it is used
Provides the information retrieved from encoder states during cross-attention.
How deep it goes
Explains values through their weighted-sum role in producing context vectors, without descending into parameter-level implementation.
Examples
Uses encoder hidden states and decoder prediction steps as the concrete artifacts for explaining value aggregation.
What you can do after
Evaluate/understand how attention weights select and combine encoder information for each decoder output.

Read how Build AI Drug Discovery Pipelines explains value →

Applied Reinforcement Learning

A value is an attention vector containing the information contributed by an input element after attention weights are computed.

How it is used
Combines input-element information according to attention weights to form each element’s attention output.
How deep it goes
Defines values as vectors and explains their weighted-sum contribution to attention outputs.
Examples
Uses building zones as input elements and considers scaling attention from 8 zones to 80.
What you can do after
Understand how attention weights select and combine information from relevant neighboring zones.

Read how Applied Reinforcement Learning explains value →

Deep Learning with PyTorch, Second Edition

The transformed representation vectors that are combined using attention weights to produce the output for each position.

How it is used
Uses transformed content vectors to integrate context into each sequence position’s attention output.
How deep it goes
Explains linear value projection and weighted summation, while treating query-key weight calculation as the surrounding attention mechanism.
Examples
Demonstrates character embeddings, next-character prediction, and a PyTorch calculation using F.linear, softmax, and matrix multiplication.
What you can do after
Apply via PyTorch by projecting inputs into values and combining them with attention weights.

Read how Deep Learning with PyTorch, Second Edition explains value →

Build a Large Language Model (From Scratch)

The attention content vector combined according to attention weights to form the context output.

How it is used
Carries transformed token content into the weighted sum that produces each self-attention context vector.
How deep it goes
Explains W_value projection, tensor shapes, and weighted combination, stopping at a single-query example rather than multi-head construction.
Examples
Uses the six-word sentence “Your journey starts with one step,” second-input query x_2, and PyTorch tensors shaped [6, 2].
What you can do after
Build it from scratch in PyTorch by projecting inputs with W_value and combining values with attention weights.

Read how Build a Large Language Model (From Scratch) explains value →

Deep Learning with Python, Third Edition

How it is used
Uses projected content vectors as the inputs combined by attention scores to produce an output.
How deep it goes
Defines values through the weighted-sum operation and contrasts them with keys, without deriving the projections.
Examples
Uses photo retrieval to compare query terms, photo tags, and returned images.
What you can do after
Understand how attention weights combine value vectors into an output.

Read how Deep Learning with Python, Third Edition explains value →

Understanding AI

A value vector is an attention vector that carries the information contributed by a token after attention weights are applied.

How it is used
Carries token information into the context-aware representation produced by attention.
How deep it goes
Explains value vectors through roles, dot-product scores, softmax weights, and weighted summation without deriving the mathematics.
Examples
Uses the sentence about an unbelievable performance and pronoun “it,” with illustrative attention weights.
What you can do after
Understand how attention combines token information into an enriched representation.

Read how Understanding AI explains value →

Grokking AI Algorithms, Second Edition

A value vector is the vector representation carrying the content contributed by a token in an attention computation.

How it is used
Uses token content to build context-rich representations during self-attention.
How deep it goes
Explains projection, weighting, elementwise scaling, and summation, stopping short of deeper training mechanics.
Examples
Works through token 0’s four-dimensional Value vector, weighted results, and final context vector.
What you can do after
Apply it via Python by projecting embeddings into V and multiplying V by attention weights.

Read how Grokking AI Algorithms, Second Edition explains value →

Rearchitecting LLMs

A value vector is the attention representation combined according to attention weights to produce an attended output.

How it is used
Describes the attention representation combined using attention weights to produce an attended output.
How deep it goes
Defines weighted value aggregation and identifies GQA projection sharing, but does not detail the computation.
Examples
Uses Llama-3.2-1B projections, where v_proj maps 2048 input features to 512 output features.
What you can do after
Understand how GQA shares value projections across heads within a group.

Read how Rearchitecting LLMs explains value →

Also covered in

What to know first

Related concepts

Questions about value

What is value?

In attention, a value is a projected vector representation of content that is combined with other values according to attention weights to produce an output.

Which Manning books explain value?

Timeless Algorithms, The Foundational Ideas, Transformers in Action, Build a DeepSeek Model (From Scratch), Build a Text-to-Image Generator (from Scratch), Sutskever's List, CUDA for LLMs, Learn Generative AI with PyTorch, Build AI Drug Discovery Pipelines and 7 more.

What should I know before learning value?

Manning books that teach value usually introduce transformer architecture, softmax, embedding and recurrent neural network first.

What is value part of?

Value is part of key-value cache. A key-value cache stores attention keys and values computed for earlier tokens so they can be reused during subsequent autoregressive generation.

What is value also called?

Value is also called V.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.