liveBook · Manning

Value

As explained in Build AI Drug Discovery Pipelines

A value vector is the attention representation combined with weights to form an attended output.

A value is an encoder hidden state that carries information into an attention output. Cross-attention combines these values according to attention weights produced from a decoder query and encoder keys. section 12.3.1

The attention computation proceeds as follows:

  • Compare the decoder query with encoder keys to produce raw attention scores. section 12.3.1
  • Normalize the scores with softmax to obtain attention weights that sum to 1. section 12.3.1
  • Compute a weighted sum of the value vectors to produce a context vector for the current prediction step. section 12.3.1
Figure 12.8 The mechanics of cross-attention. A decoder state acts as a query (qj) and compares itself against all encoder states (keys, K). These comparisons generate attention weights (α), which are used to compute a weighted sum of the encoder states (values, V), creating a context vector (cj) specific to the current prediction step.

Trace how a decoder state becomes a query, attends to encoder states, and combines their values into a prediction-specific context vector. section 12.3.1

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains values through their weighted-sum role in producing context vectors, without descending into parameter-level implementation.
Examples
Uses encoder hidden states and decoder prediction steps as the concrete artifacts for explaining value aggregation.
What you can do after
Evaluate/understand how attention weights select and combine encoder information for each decoder output.
How it connects
Connects values to cross-attention, encoder hidden states, queries, keys, attention weights, softmax, and context vectors.

Also explained in

Compare how 15 books explain value →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.