liveBook · Manning

Value

As explained in Deep Learning with Python, Third Edition

In attention, a value is a projected vector representation of content that is combined with other values according to attention weights to produce an output.

In attention, a value is a projected vector carrying content into the output. Attention scores determine how much each value contributes to the weighted sum. Values can differ from keys, which are used to score relevance. section 15.3.1

A search analogy

Imagine searching a photo database: the query matches photo tags (keys), and the matching scores weight the photos (values) returned. section 15.3.1

Figure 15.6 Retrieving images from a database: the query is compared to a set of keys, and the match scores are used to rank values (images).
query key value

The diagram illustrates queries matched against keys, with scores ranking the image values. section 15.3.1

How Deep Learning with Python, Third Edition covers it

How deep it goes
Defines values through the weighted-sum operation and contrasts them with keys, without deriving the projections.
Examples
Uses photo retrieval to compare query terms, photo tags, and returned images.
What you can do after
Understand how attention weights combine value vectors into an output.
How it connects
Connects values to attention, queries, keys, and dot-product scoring.

Also explained in

Compare how 15 books explain value →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.