A token budget is a system-set limit on the tokens allocated to a model call. It is a softer ceiling than the model’s context window, chosen with cost and latency targets in mind and normally below the window limit. section 7.1
The context window is the hard maximum a call can carry. The attention budget is a separate constraint: adding content spreads the model’s attention across more tokens and can reduce the salience of important instructions, evidence, and state. Too little context can produce vague answers; too much adds noise, latency, and cost. section 7.1 section 1.1.1
Maximum output tokens is a distinct inference-time setting that caps how many tokens the model may generate. Reasoning tokens also consume the context window and can increase token usage and response time. section 1.2.1
Context management works within these limits through four complementary techniques:
- Selection chooses relevant tokens for the active prompt. section 7.1
- Writing persists recoverable state outside the prompt. section 7.1
- Compression condenses verbose history. section 7.1
- Isolation divides complex tasks across separate context windows. section 7.1
Customer support assistant
For an assistant answering order questions, the budget guides how much relevant information enters a call. Selection, external state, and compressed history can keep the active context focused without treating the full window as the target. section 7.1
Figure 7.1 Context is managed through four complementary techniques: selection chooses relevant tokens for the active prompt, writing persists state to external storage, compression condenses verbose history, and isolation divides complex tasks across separate context windows
The figure maps the four context-management techniques to the resources they manage. section 7.1











