liveBook · Manning

Query

As explained in Build AI Drug Discovery Pipelines

A query vector is an attention representation that specifies the information sought from a set of keys and values.

A query vector is the decoder’s current hidden state: it represents the information currently sought from encoder hidden states during cross-attention. section 12.3.1

Cross-attention uses the query in a short retrieval workflow:

  • Compare the query with every encoder hidden state, treated as a key, to produce compatibility scores or logits. section 12.3.1
  • Pass the scores through softmax to obtain normalized attention weights over the encoder positions. section 12.3.1
  • Use those weights to combine the encoder hidden states, treated as values, into a context vector for the current output. section 12.3.1

Protein-sequence decoding

While generating an output from encoded protein information, each decoding step produces a query that selects the most relevant encoder positions rather than relying on one fixed context vector. section 12.3.1

Figure 12.8 The mechanics of cross-attention. A decoder state acts as a query (qj) and compares itself against all encoder states (keys, K). These comparisons generate attention weights (α), which are used to compute a weighted sum of the encoder states (values, V), creating a context vector (cj) specific to the current prediction step.

The cross-attention diagram traces a decoder query through key comparisons, attention weights, and the resulting context vector. section 12.3.1

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains query-key scoring, softmax weighting, and value aggregation, without detailing a specific scoring formula.
Examples
Uses decoder steps and a cross-attention diagram for protein-sequence information retrieval.
What it assumes
Assumes encoder and decoder hidden states, while defining how those states participate as keys, values, and queries.
What you can do after
Understand how a decoder selects targeted encoder information through query-key comparisons and weighted value aggregation.
How it connects
Connects queries to cross-attention, keys, values, attention weights, softmax, encoder hidden states, and context vectors.

Also explained in

Compare how 14 books explain query →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.