Caching stores results so repeated requests can return them without repeating the original retrieval or computation. For agent tools, caching is safest when calls are idempotent: the same inputs produce the same result, whether called once or repeatedly. A weather lookup is idempotent; sending an email is not. section 8.3.4
Common caching opportunities in agent systems include:
- Cache idempotent tool-call results so retries and replays can return prior results without repeating side effects. section 8.3.4
- Cache repeated research queries, but clear the cache regularly to reduce the risk of stale search results. section 11.4
- Consider prompt, response, and embedding caches for repeated work; plan how each cache will be invalidated because stale entries can produce stale behavior. section 8.3.8
Listing 8.6 06_idempotent_key_example.py
from fastapi import FastAPI
from pydantic import BaseModel
import hashlib, json, asyncio
app = FastAPI(title="Idempotent-by-Inputs Tool Proxy")
class ToolIn(BaseModel):
name: str
args: dict
CACHE: dict[str, dict] = {} # demo only
def cache_key_from_inputs(name: str, args: dict) -> str: #1
"""
Build a deterministic key directly from function inputs.
Canonicalize JSON to ensure stable ordering of dict keys.
"""
canon = json.dumps({"name": name, "args": args},
sort_keys=True, separators=(",", ":"), ensure_ascii=False)
return hashlib.sha256(canon.encode("utf-8")).hexdigest()
@app.post("/tool")
async def call_tool(body: ToolIn): #2
key = cache_key_from_inputs(body.name, body.args) #3
if key in CACHE:
return {"cached": True, "result": CACHE[key], "key": key} #4
# Fake tool execution (replace with real call)
await asyncio.sleep(1.0) #5
result = {"ok": True, "tool": body.name, "echo": body.args}
CACHE[key] = result #6
return {"cached": False, "result": result, "key": key}
See an example that builds a deterministic key from tool inputs, checks a cache for that key, and returns a cached result when available. section 8.3.4
Repeated research queries
A research agent can reuse cached results for queries it has already run, reducing repeated work. Regular cache clearing helps limit stale results. section 11.4
