A context window is the maximum number of tokens a model can process at once. Its active input may include a request, instructions, examples, conversation turns, retrieved documents, and generated tokens. Material outside the window cannot shape the next prediction—not because it is irrelevant, but because it is not available for that generation step. A surrounding system may need to omit, truncate, or compress it. section 6.4.2
The window is not a persistent memory. Learned patterns in model parameters provide background, while the prompt supplies task-specific information at inference time. Fluent responses can still be unreliable when relevant context is missing or displaced. section 6.4.2 section 6.0
Tokenization and window size shape how much information can be processed:
- Tokens, not words, measure the window. In English, one token often corresponds to about three quarters of a word. section 6.4.2
- Finer tokenization can preserve nuance but lengthens sequences; larger units use space more efficiently but can lose detail. section 2.4.6
- Widely deployed conversational systems in 2022 had roughly 4,000–8,000-token windows; around 128,000 tokens became available in major production models by late 2023 and 2024. section 6.4.2
A limited field of view
A document set may contain many relevant details, but only the portion that fits in the window is available to shape a particular response. section 6.4.2





































