liveBook · Manning

Context window

As explained in Understanding AI

A context window is the bounded amount of input and generated text that a model can access simultaneously when producing an output.

Have you ever asked an AI to keep a detail consistent across a long chat, only to find that it seems to have forgotten it? A context window is the bounded set of text a model can access while producing its next output. It includes the prompt, earlier conversation messages, and text the model has already generated. section 2.1.4 section 2.3

Think of it as a fixed-size desk. The model can use everything currently on the desk at once, because those tokens are available to its attention mechanism. But when the desk is full, older material may fall off to make room. Text outside the window is not faintly remembered or merely harder to retrieve; it is unavailable to the model. section 2.1.4 section 2.3

A lost opening tone

Suppose a concert review begins, “The performance was unbelievable — the crowd couldn't believe it.” Much later, you ask for an ending that matches the opening's enthusiastic tone. If the opening remains in the context window, the model can use its wording and emotional energy. If it has fallen outside the window, the ending may be generic or tonally inconsistent because the model can no longer see the original line. section 2.1.4

This limit can quietly change a long conversation. A request may look clear in the latest message, yet depend on names, constraints, notes, or stylistic choices supplied much earlier. In a long-form writing exchange, losing earlier material can make a model omit a central person or fail to reproduce the intended symmetry between an opening and a closing. section 2.2.4 section 2.0

Fitting inside the window is necessary, but it does not guarantee equal emphasis. Models may attend more strongly to material at the beginning and end of a long input, while information in the middle can receive less attention. Where important information appears can therefore matter as well as whether it remains visible. section 2.1.4

Figure 2.4 The context window defines the boundary of what the model can see. Tokens inside it are fully available to the attention mechanism. Tokens outside it are gone. In a long conversation, early messages can silently fall outside the window, taking their context with them.

The figure shows the central boundary: tokens inside the context window remain available to attention, while earlier tokens that fall outside are gone from the model's view. section 2.1.4

A context window is a limit on what the model can see at one moment, not evidence that the model understands or permanently remembers a conversation. Recognizing that boundary helps distinguish a wording problem from a visibility problem when a response loses the thread. section 2.0 section 2.3

How Understanding AI covers it

How deep it goes
Defines the window through tokens and attention, using desk and conversation examples without mathematical treatment.
Examples
Uses a concert-review opening and a long-form climate-scientist profile whose early details disappear.
What it assumes
Introduces the limit after tokens, embeddings, and attention, while explaining its practical behavior in plain language.
What you can do after
Evaluate/understand whether a lost detail reflects a context-window visibility limit rather than a failed instruction.
How it connects
Connects context windows to tokens, embeddings, attention, prompts, and conversational coherence.

Also explained in

Compare how 38 books explain context window →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Understanding AI. The section links point to the parts of the book it draws on.