liveBook · Manning

Trajectory

As explained in Designing AI Agents

A record of events and operations during a system run, arranged to show its progression and support diagnosis.

An agent trajectory is the ordered record of a run: model calls, proposals, policy decisions, tool responses, and observed state changes. It connects what the agent attempted with what the system actually did, making the run available for diagnosis and review. section 9.5.1

A useful trace correlates the user, agent identity, workload, session, run, tool call, approval, and result across services. Preserve input references, versions, and identities alongside the sequence. Replaying a prompt alone is insufficient: model settings, session context, and external tool data may have changed. section 9.5.1

Table 9.7 The agent observability stack. Traditional observability extended with evaluations and replays.
Pillar
Traditional software
Agent systems
Logs
Application events, errors
Tool calls, model outputs, policy decisions
Metrics
Latency, throughput, error rate
Task outcomes, cost, overrides, tool selection accuracy
Traces
Request → service → database
Agent trajectory: context → proposal → policy → tool result → observed effect
Evaluations (new)
Unit tests, integration tests
Outcome tests, LLM-as-judge, human review
Replays (new)
Request or event replay
Replay stored trajectories with versioned context

Compare trajectories with logs, metrics, evaluations, and replay evidence; each answers a different question about a run. section 9.5.1

Diagnosing an excessive coupon

A successful coupon tool call does not establish that the compensation was authorized. Inspect the proposal, the messages the model saw, the case limit, earlier coupons, and the policy decision. An excessive proposal points toward input or planning; a gate that admitted it also implicates policy or task-state input. Turn the incident into regression cases for both excessive and valid requests, linking new traces and their versions to the incident. section 9.5.6

In collaborative runs, inspect the seams as well as the final result: rejected handoffs, duplicate work, contradictions, blocked dependencies, and integration rework can localize coordination failures. For consequential work, attach the claim, source, artifact version, acceptance criterion, decision owner, and integration receipt. A trace can reveal misalignment, but it cannot prove reviewer independence or final correctness. section 8.6

Build the execution record from observable behavior, not an assumed account of the model's reasoning. A trace can show what happened, but not prove why the model thought it was right. At production volume, use structured alerts and stratified sampling by task difficulty, tenant, and risk; high-risk paths may require complete retention. section 9.5.7 section 5.9

A raw trace can also support learning without becoming an instruction. Keep the execution, outcome, tool evidence, and environment version for audit; compress it into a candidate lesson with provenance, then admit a scoped heuristic only with sufficient support. A replay record alone grants no authority to execute a tool call or bypass a gate. section 7.5.1

How Designing AI Agents covers it

How deep it goes
Defines trajectory contents and correlation requirements, then details replay conditions, failure diagnosis, sampling, and fidelity limits.
Examples
Traces an excessive coupon through its proposal and policy gate, and localizes a lost identifier to a model-call handoff.
What you can do after
Diagnose agent failures by following evidence through proposals, policy decisions, tool responses, and resulting state.
How it connects
Connects trajectories to observability, policy decisions, regression testing, experience replay, and governance.

Also explained in

Compare how 25 books explain trajectory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.