Value

As explained in Deep Learning with Python, Third Edition

In attention, a value is a vector representation of input content that is combined with other values according to attention weights to produce an output.

In attention, a value is a content vector used in the weighted sum that produces an output. It can differ from the vector used to score relevance. section 15.3.1

Figure 15.6 Retrieving images from a database: the query is compared to a set of keys, and the match scores are used to rank values (images).

Photo retrieval offers an analogy: a query matches against keys, while the resulting scores rank the values—the photos being retrieved. section 15.3.1

How Deep Learning with Python, Third Edition covers it

How deep it goes

Explains weighted summation and separate value projections with parameterized attention code.

Examples

Uses photo retrieval as an analogy and projected source vectors in parameterized attention code.

What you can do after

Understand how attention scores weight value vectors to produce an output.

How it connects

Connects values to queries and keys, distinguishing vectors used for matching from vectors used for aggregation.

liveBookmanning.com

Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Transformers in ActionIn attention, a value is a vector representation containing the content aggregated according to attention weights.
• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)A value is an attention vector whose information is combined according to attention weights.
• 

  liveBookmanning.com
  Sutskever's List
• 

  liveBookmanning.com
  Learn Generative AI with PyTorch
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionThe transformed representation vectors that are combined using attention weights to produce the output for each position.
• 

  liveBookmanning.com
  Build a Large Language Model (From Scratch)The attention content vector combined according to attention weights to form the context output.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA value vector is the vector representation carrying the content contributed by a token in an attention computation.

Compare how 8 books explain value →

Related concepts

• key-value cachepart of · A key-value cache is a data structure that stores previously computed transformer attention keys and values for reuse during subsequent token generation.
• attentionused by · Attention is a neural-network mechanism that computes input-dependent weights to combine information according to its relevance to a computation.
• attention weightused by · An attention weight is a computed coefficient, often normalized, that determines how strongly an input element or value contributes to an attention-based representation.
• grouped-query attentionused by · Grouped-query attention partitions query heads into groups whose members share key and value projections, reducing attention computation and memory use.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.