liveBook · Manning

Value

As explained in Deep Learning with R, Third Edition

A value is the representation that attention combines after relevance has been computed from queries and keys.

A value is the representation that attention combines after query-key relevance has been computed. Attention weights determine how much of each value contributes to the output. section 15.3.1

Database lookup analogy

For photo search, the query is the search term, keys are photo tags used for matching, and values are the retrieved photos. Attention similarly uses match scores to combine values. section 15.3.1

Parameterized attention can project values into a separate space before the weighted sum, then transform the combined output afterward. section 15.3.1

How Deep Learning with R, Third Edition covers it

How deep it goes
Defines values, contrasts them with keys, and demonstrates separate value projections without deriving the underlying mathematics.
Examples
Uses a photo-search analogy and R implementations that calculate weighted sums from projected values.
What it assumes
Assumes attention terminology, while distinguishing values from queries and keys through a search analogy.
What you can do after
Apply it via R attention implementations that project values and combine them using learned relevance weights.
How it connects
Connects values to attention, queries, keys, dot-product attention, softmax, and multi-head attention.
Cover of Deep Learning with R, Third Edition
liveBookmanning.com
Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 7 books explain value →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.