Context optimization deliberately transforms the context sent to a language model so an agent remains efficient and task-relevant. It addresses context explosion, where accumulated context consumes a large share of the model window or exceeds it during complex problem solving. section 6.0 section 6.2.3
The central design separates storage from presentation: retain execution history while changing how the model sees it. A before_llm_callback reads ExecutionContext and modifies LlmRequest.contents immediately before the model call; ExecutionContext.events remains read-only. section 6.2.1
Common optimization strategies include:
- Apply a sliding window to retain only a bounded portion of conversation history. section 6.0
- Compact or summarize accumulated context when it becomes too large. section 6.0
- Trigger optimization by token threshold rather than waiting for the model limit. section 6.2.1 section 6.2.3
Threshold-based optimizer
A callback factory counts tokens, compares the request with a configured threshold, and invokes a synchronous or asynchronous optimization function only when the threshold is exceeded. Optimization functions receive (context, request) and can record compression progress in context.state. section 6.2.1
Listing 6.4 Token counting with tiktoken
def count_tokens(request: LlmRequest) -> int:
"""Calculate total token count of LlmRequest"""
import tiktoken
from .llm import build_messages
# Select encoding for model, use default on failure
try:
encoding = tiktoken.encoding_for_model(request.model_id or "gpt-5")
except KeyError:
encoding = tiktoken.get_encoding("o200k_base")
# Convert to API message format then count tokens
messages = build_messages(request)
total_tokens = 0
for message in messages:
# Per-message overhead (role, separators, etc.)
total_tokens += 4
# Content tokens
if message.get("content"):
total_tokens += len(encoding.encode(message["content"]))
# tool_calls tokens
if message.get("tool_calls"):
for tool_call in message["tool_calls"]:
func = tool_call.get("function", {})
if func.get("name"):
total_tokens += len(encoding.encode(func["name"]))
if func.get("arguments"):
total_tokens += len(encoding.encode(func["arguments"]))
# Tool definition tokens
if request.tools:
for tool in request.tools:
tool_def = tool.tool_definition
total_tokens += len(encoding.encode(json.dumps(tool_def)))
return total_tokens
Inspect a token counter that builds API messages and counts instructions, contents, tool definitions, and tool-call tokens. section 6.2.3
