A token limit caps token usage for a request or session. It can be enforced before a model call in middleware or applied as a per-request or per-session quota. section 2.4.1 section 4.3.4
Use token limits as one layer of resource control:
- Pair quotas with per-user or API-key rate limits, timeouts, and hard provider spend limits; budget alerts alone do not stop resource use. section 4.3.4
- Monitor token consumption and cost, and consider circuit breakers that pause execution before costs escalate. section 4.3.4
Runaway requests
A QA script sent thousands of requests in a loop and incurred several hundred dollars in charges. Per-session token caps were added alongside rate limits and hard spend limits. section 4.9











