liveBook · Manning

Trajectory

As explained in Applied Reinforcement Learning

A trajectory is a sampled sequence of states, actions, and outcomes generated by an agent interacting with an environment.

A trajectory is a complete sequence of states, actions, and rewards generated as a policy interacts with an environment. Its discounted return is the sum of its rewards, weighted by time. section 9.2

Policy quality is the expected cumulative reward across possible trajectories. Because a policy influences actions, visited states, and rewards, its parameters shape which trajectories occur. section 9.2

In REINFORCE, trajectories provide the learning signal:

  • Collect complete trajectories, compute their returns, and update the policy to make high-return trajectories more probable. section 9.8
  • Subtracting a state-value baseline from the return measures how much better or worse an action performed than expected, reducing variance. section 9.8

How Applied Reinforcement Learning covers it

How deep it goes
Defines state-action-reward sequences and discounted returns, then connects them to policy gradients; details trajectory collection only for REINFORCE.
What you can do after
Apply it via REINFORCE by collecting complete trajectories, computing returns, and increasing the probability of high-return runs.
How it connects
Connects trajectories to policies, rewards, returns, policy gradients, and REINFORCE.

Also explained in

Compare how 25 books explain trajectory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.