Have you ever asked an AI to keep a detail consistent across a long chat, only to find that it seems to have forgotten it? A context window is the bounded set of text a model can access while producing its next output. It includes the prompt, earlier conversation messages, and text the model has already generated. section 2.1.4 section 2.3
Think of it as a fixed-size desk. The model can use everything currently on the desk at once, because those tokens are available to its attention mechanism. But when the desk is full, older material may fall off to make room. Text outside the window is not faintly remembered or merely harder to retrieve; it is unavailable to the model. section 2.1.4 section 2.3
A lost opening tone
Suppose a concert review begins, “The performance was unbelievable — the crowd couldn't believe it.” Much later, you ask for an ending that matches the opening's enthusiastic tone. If the opening remains in the context window, the model can use its wording and emotional energy. If it has fallen outside the window, the ending may be generic or tonally inconsistent because the model can no longer see the original line. section 2.1.4
This limit can quietly change a long conversation. A request may look clear in the latest message, yet depend on names, constraints, notes, or stylistic choices supplied much earlier. In a long-form writing exchange, losing earlier material can make a model omit a central person or fail to reproduce the intended symmetry between an opening and a closing. section 2.2.4 section 2.0
Fitting inside the window is necessary, but it does not guarantee equal emphasis. Models may attend more strongly to material at the beginning and end of a long input, while information in the middle can receive less attention. Where important information appears can therefore matter as well as whether it remains visible. section 2.1.4
Figure 2.4 The context window defines the boundary of what the model can see. Tokens inside it are fully available to the attention mechanism. Tokens outside it are gone. In a long conversation, early messages can silently fall outside the window, taking their context with them.

The figure shows the central boundary: tokens inside the context window remain available to attention, while earlier tokens that fall outside are gone from the model's view. section 2.1.4
A context window is a limit on what the model can see at one moment, not evidence that the model understands or permanently remembers a conversation. Recognizing that boundary helps distinguish a wording problem from a visibility problem when a response loses the thread. section 2.0 section 2.3





































