liveBook · Manning

Context vector

A context vector is a numerical vector that summarizes information from relevant input context for a model’s prediction or subsequent processing.

How 3 Manning books explain it

Build a DeepSeek Model (From Scratch)

A context vector is a numerical representation summarizing information relevant to a model's prediction at a particular sequence position.

How it is used
Uses the newest context vector to motivate efficient next-token inference and the key-value cache.
How deep it goes
Derives context matrices from attention weights and values, then traces the newest vector back to its query, keys, and values.
Examples
Works through “The next day is” and its generated token “bright” using explicit matrix dimensions.
What you can do after
Evaluate how attention constructs a context vector and why cached keys and values reduce repeated inference work.

Read how Build a DeepSeek Model (From Scratch) explains context vector →

Sutskever's List

A context vector is a weighted combination of source representations that summarizes the input information selected by an attention mechanism at a decoding step.

How it is used
Summarizes source information selected by attention at each decoding step in translation.
How deep it goes
Defines the context vector as a weighted source summary but does not derive or implement its computation.
Examples
Illustrates attention aligning “zone” with “Area” before revisiting earlier words in “European Economic Area.”
What you can do after
Understand how attention selects source annotations while generating each target word.

Read how Sutskever's List explains context vector →

Build AI Drug Discovery Pipelines

A context vector is the weighted sum of value representations produced by an attention operation for a particular output step.

How it is used
Uses attention to give each prediction access to relevant information across a long protein sequence.
How deep it goes
Defines the weighted-sum result and relates it to encoder states, queries, keys, values, and attention weights.
Examples
Uses a 500-amino-acid protein and hidden states from positions 50, 200, and 400 to illustrate distributed sequence information.
What you can do after
Understand why output-specific weighted summaries preserve information that a single final hidden state can lose.

Read how Build AI Drug Discovery Pipelines explains context vector →

Also covered in

What to know first

Related concepts

Questions about context vector

What is context vector?

A context vector is a numerical vector that summarizes information from relevant input context for a model’s prediction or subsequent processing.

Which Manning books explain context vector?

Build a DeepSeek Model (From Scratch), Sutskever's List and Build AI Drug Discovery Pipelines.

What should I know before learning context vector?

Manning books that teach context vector usually introduce recurrent neural network, softmax, LSTM and transformer architecture first.

What is context vector a kind of?

Context vector is a kind of vector. A vector is an ordered collection of components representing an element of a vector space.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.