liveBook · Manning

Token leverage

As explained in Designing AI Agents

The ratio of agent output quality to the token cost required to produce it.

Token leverage is the ratio of agent output quality to the token cost of producing it. It helps teams evaluate agent designs alongside reliability and latency, especially as usage scales. section 1.5

More context does not necessarily improve leverage. Irrelevant information consumes tokens without adding evidence that can change a decision, and it can degrade performance. Extra uncached context is paid for again on each call. section 3.1.2

When more context hurts

A full API specification can lead an agent to hallucinate endpoints from irrelevant sections. Similarly, three weeks of Git history can bury the bug under investigation. Removing irrelevant context can reduce waste, but compare runs only when task outcome, review quality, and safety are held fixed. section 3.1.2

Table 3.1 The perception engineer's trilemma
Dimension
Constraint
Consequence
Token budget
Context windows have finite, model- and configuration-dependent capacity. A 50-cycle task with one 2K-token tool result per cycle consumes 100K tokens before counting anything else.
You cannot include everything.
Attention quality
Even within the budget, attention degrades with length. Hong et al. (2025) tested 18 LLMs and found "increasingly unreliable performance as input length grows" [3].
More context often means worse reasoning.
Token cost
Uncached input is processed again on each call, while eligible prefixes may receive a lower cached rate. Input-heavy agent workloads therefore make context reuse economically significant [4].
Wasteful perception drains your budget.

The perception trilemma connects token budgets, attention quality, and the recurring cost of uncached input. section 3.1.2

How Designing AI Agents covers it

How deep it goes
Defines the quality-to-cost ratio and illustrates how irrelevant context affects both sides of the trade-off.
Examples
Illustrates wasted context with a full API specification, three weeks of Git history, and 150K irrelevant tokens.
What you can do after
Evaluate agent designs by comparing leverage with task outcome, review quality, and safety held fixed.
How it connects
Connects context management and attention quality with token cost and agent reliability.

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.