A trajectory is the ordered record of an agent’s interaction with an environment: τ = (s0, a0, s1, a1, ...). Each step pairs the current state with an action and leads to another state. Trajectories, also called episodes or rollouts, provide the experience used to judge a policy through cumulative reward. section 12.1.2
From treasure hunting to CartPole
In Reach the Treasure, an agent starts in state 2 and chooses left or right until it reaches treasure, falls into a fire pit, or exits. A resulting trajectory records each visited state and selected action, while rewards of +10, −10, or +4 indicate the outcome. In CartPole, the same idea becomes a longer interaction: each trajectory contains observations, sampled actions, rewards, and termination information while the agent tries to keep the pole upright. section 12.1.2 section 12.3
A practical rollout loop records experience step by step:
- Reset the environment and obtain the initial state or observation. section 12.3
- Use the policy to produce action probabilities, sample an action, and advance the environment. section 12.3
- Store the reward and next state, update learning statistics, and stop when the episode ends. section 12.3
Figure 12.3 An agent learns through interaction with the environment.
Use the interaction diagram to connect states, actions, rewards, and environment transitions into a complete rollout. section 12.1.2
