A rollout is one complete trajectory of interaction between a reinforcement-learning agent and its environment, beginning at an initial state and ending when the environment terminates. The trajectory records the agent’s successive states, actions, rewards, and transitions. section 11.1.1 section 11.5
A rollout typically follows this operational pattern:
- Reset the environment to obtain the initial state and clear the termination flag. section 11.1.1 section 11.5
- Select an action from the current state, using either exploration or the agent’s current policy. section 11.1.1 section 11.5
- Step the environment, collect the next state, reward, and termination status, then update the agent. section 11.1.1 section 11.5
- Stop when the agent reaches a goal, encounters failure, or otherwise produces a terminal transition. section 11.1.1 section 11.5
Frozen Lake rollout
A rollout starts on the 4-by-4 Frozen Lake grid. The Q-learning agent repeatedly chooses up, down, left, or right, receives rewards after each move, and stops when it reaches the goal or falls through a hole. Several completed rollouts can then be used to exercise the learned agent. section 11.1.1
Use the Frozen Lake diagram to inspect the environment in which a rollout moves from its starting state toward success or failure. section 11.1.1
