A failed agent response is hard to fix if all you can see is the user’s complaint. Monitoring makes the run inspectable. Logging records inputs, outputs, and intermediate steps so a failure can be reviewed, queried, and replayed. It must include evaluator activity: an evaluator that approves bad output can otherwise fail silently. section 7.3.4
Logs answer what was recorded; traces connect activity across a turn; metrics show patterns across many turns. Together, they provide observability from the user interface through the gateway, agent, tools, and model. section 8.3.6
Build a useful monitoring trail in layers:
- Start with basic logging, tests, and guardrails. Record prompts, tool calls, token counts, latency, and outcomes, including interactions with supporting services. section 11.1.5
- Add a trace span for each turn and tool call. Carry session_id, turn_id, and tool_call_id so related activity can be found together. section 8.3.6
- Track operational measures such as latency, tool success, errors, and cost; quality measures such as evaluator pass rate and user feedback; and product measures such as task completion. section 8.3.6
- Keep structured JSON logs and sample debugging payloads, but redact personally identifiable information. section 8.3.6
Listing 8.7 07_phoenix_metadata.py
from agents import trace
from phoenix.otel import register
from openinference.instrumentation import using_session, using_user, using_metadata
register(project_name="Agents In Action") #1
with (
using_session("s-123"),
using_user("u-42"),
using_metadata({"turn_id":"t-1","intent":"generate_image"})):
with trace("agent-turn"): #2
pass #3
# ... your agent code here ...
This tracing setup attaches a session, user, turn ID, and intent to an agent turn. Those attributes make related activity filterable; the placeholder marks where agent code would run. section 8.3.6
Use the trail to investigate incidents and improve the system, not merely to confirm that it is running. Operational metrics show system behavior; quality and product metrics show whether that behavior meets its purpose. Monitoring can also be a separate concern inside a modular cognitive agent, alongside reasoning, memory, and adaptation. section 8.3.6 section 10.4.3
