liveBook · Manning

Token limit

As explained in Context Engineering

A token budget is a limit on the tokens allocated to an AI system or request.

A token budget is a system-set limit on the tokens allocated to a model call. It is a softer ceiling than the model’s context window, chosen with cost and latency targets in mind and normally below the window limit. section 7.1

The context window is the hard maximum a call can carry. The attention budget is a separate constraint: adding content spreads the model’s attention across more tokens and can reduce the salience of important instructions, evidence, and state. Too little context can produce vague answers; too much adds noise, latency, and cost. section 7.1 section 1.1.1

Maximum output tokens is a distinct inference-time setting that caps how many tokens the model may generate. Reasoning tokens also consume the context window and can increase token usage and response time. section 1.2.1

Context management works within these limits through four complementary techniques:

  • Selection chooses relevant tokens for the active prompt. section 7.1
  • Writing persists recoverable state outside the prompt. section 7.1
  • Compression condenses verbose history. section 7.1
  • Isolation divides complex tasks across separate context windows. section 7.1

Customer support assistant

For an assistant answering order questions, the budget guides how much relevant information enters a call. Selection, external state, and compressed history can keep the active context focused without treating the full window as the target. section 7.1

Figure 7.1 Context is managed through four complementary techniques: selection chooses relevant tokens for the active prompt, writing persists state to external storage, compression condenses verbose history, and isolation divides complex tasks across separate context windows
context management v6

The figure maps the four context-management techniques to the resources they manage. section 7.1

How Context Engineering covers it

How deep it goes
Defines the budget and contrasts it with context and attention limits, without prescribing numerical settings.
Examples
Uses a customer support assistant answering order questions to frame context management.
What you can do after
Evaluate context allocation by balancing relevance, reasoning quality, cost, and latency.
How it connects
Connects token limits to context windows, attention budgets, context management, and maximum output tokens.

Also explained in

Compare how 12 books explain token limit →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Context Engineering. The section links point to the parts of the book it draws on.