liveBook · Manning

Context window

As explained in Machines that Think

A context window is the bounded amount of input information that a model can process or retain at one time.

A context window is the maximum number of tokens a model can process at once. Its active input may include a request, instructions, examples, conversation turns, retrieved documents, and generated tokens. Material outside the window cannot shape the next prediction—not because it is irrelevant, but because it is not available for that generation step. A surrounding system may need to omit, truncate, or compress it. section 6.4.2

The window is not a persistent memory. Learned patterns in model parameters provide background, while the prompt supplies task-specific information at inference time. Fluent responses can still be unreliable when relevant context is missing or displaced. section 6.4.2 section 6.0

Tokenization and window size shape how much information can be processed:

  • Tokens, not words, measure the window. In English, one token often corresponds to about three quarters of a word. section 6.4.2
  • Finer tokenization can preserve nuance but lengthens sequences; larger units use space more efficiently but can lose detail. section 2.4.6
  • Widely deployed conversational systems in 2022 had roughly 4,000–8,000-token windows; around 128,000 tokens became available in major production models by late 2023 and 2024. section 6.4.2

A limited field of view

A document set may contain many relevant details, but only the portion that fits in the window is available to shape a particular response. section 6.4.2

How Machines that Think covers it

How deep it goes
Explains token limits, active input, and omission or compression, without detailing implementation mechanisms.
Examples
Uses a visual-field analogy, prompt contents, and historical token-window sizes to illustrate bounded input.
What you can do after
Evaluate how missing or displaced context can shape a model response and limit its reliability.
How it connects
Links context windows to prompts, tokens, tokenization, model parameters, and inference.

Also explained in

Compare how 38 books explain context window →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.