Telemetry is the raw operational data and signals generated by a running agent and transmitted for monitoring and analysis. It enables observability, debugging, evaluation, and operational response by exposing latency, reasoning traces, tool calls, token use, and errors. section 10.1
Organize telemetry into three complementary forms:
- Metrics provide compact numerical measurements for dashboards, alerts, and performance tracking, such as tool latency, token use, task cost, and LLM error rate. section 10.1.1
- Traces show the complete journey of one request; spans represent work such as LLM calls, tool invocations, memory reads, or planning steps, while events mark instantaneous occurrences. section 10.1.1
- Logs record contextual details such as authentication failures, service events, API errors, and tool inputs or outputs, but selective logging limits data-volume growth. section 10.1.1
Instrumentation inserts collection code at important execution points, including LLM requests, tool calls, planning, and workflow steps. The resulting data is collected, processed, and exported to an observability backend, where it supports analysis of performance, reliability, cost, and behavioral quality. section 10.1.2
Tracing an agent request
For a request to plan a round-trip KTX itinerary to Busan, one trace can contain the interpretation step, LLM calls, tool invocations, returned tool outputs, retries, and final response. Metrics can record latency and token use, while logs preserve selected inputs, outputs, and failures. section 10.1.1
Table 10.1 What to instrument
| What to record |
Description |
|---|---|
| Input prompt |
The user’s request given to the agent |
| Intermediate reasoning |
The agent’s internal reasoning process (e.g., chain of thought) |
| Tool-call logs |
Which tools were called and in what order |
| Tool outputs |
The results returned by each tool |
| Final output |
The response delivered to the user |
| Runtime/cost |
Latency, number of API calls, token use, and so on |
Use the instrumentation checklist to decide which agent inputs, reasoning records, tool activity, outputs, and runtime or cost signals to capture. section 10.1.2
