liveBook · Manning

Trajectory

As explained in LLM Evaluation and Alignment, The Foundational Ideas

An execution trace is a chronological record of an agent's reasoning state, actions, retrieved results, and subsequent decisions.

A trajectory is a record of the steps a policy takes to complete a task and produce a response. For an agent, it can follow a loop of reasoning, acting, and observing results. An execution trace records those intermediate steps, making it possible to inspect how the agent arrived at its answer rather than judging the answer alone. section 6.4.1 section 7.3.1

Figure 7.3 A trajectory is the steps that the agent has taken to complete a task to produce a final response. Agents typically go through loops of reason -> act -> result to complete tasks.

The trajectory follows an agent’s reasoning, actions, and results as it works toward a final response. section 7.3.1

In a retrieval-augmented agent, log the reasoning state, query, selected tool or source, retrieved documents, and decision after each result. Evaluate whether the agent identified its information need, formed an appropriate query, chose the right source, retrieved relevant evidence, and stopped at the right time. Evaluate the final synthesis for faithfulness, relevance, and completeness. section 6.4.1

Diagnosing a search

A legal research agent may correctly identify a need for information about remedies after breach of contract but search for “breach of contract options” instead of “equitable remedies contract breach.” The trace exposes the weak query; a final-answer-only evaluation may not reveal where the failure began. section 6.4.1

In reinforcement learning, a rollout samples a prompt, follows the policy’s steps to a response, and observes a reward. A training batch collects trajectories for a policy update that increases expected reward. Trajectories can also supply short segments for human preference comparisons: labeled pairs train a reward model, whose scores support policy updates. section 7.3.1 section 7.4.2

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines rollouts and details trace logging, stepwise evaluation, and segment-based preference labeling.
Examples
Uses a legal research agent’s weak search query and short Atari or MuJoCo simulation clips compared by annotators.
What you can do after
Diagnose/debug with it by inspecting intermediate agent decisions rather than relying only on the final answer.
How it connects
Connects trajectories to policies, rewards, retrieval-augmented generation, process evaluation, and reward models.

Also explained in

Compare how 25 books explain trajectory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.