A trajectory follows an agent through a reinforcement-learning trial: from its starting state, through successive actions and states, to an outcome. A complete trial is called an episode. It can end in success, failure, or a timeout. section 10.6 section 10.11
One drive, one episode
Imagine a self-driving car trying to reach its owner. If it takes 14 actions to reach the goal, those actions belong to one episode—not 14 episodes. After each move, the car receives feedback and can update its Q-table, which estimates the value of actions in different states. section 10.6
A Q-learning training run repeats this pattern:
- Reset the simulator and read the starting state. section 10.6
- Choose an action, move, receive a reward, and update the Q-table; continue until the trial ends. section 10.6
- Start another episode. Repetition helps estimates of future rewards propagate back from the goal; even a failed run can provide useful training data. section 10.6 section 10.11
Episodes need not always begin in the same place. Starting the car at different valid parking-lot positions encourages a strategy that works across more of the environment. The same repeated-trial idea also applies when a neural network predicts action values instead of storing them in a Q-table. section 10.11
Figure 10.13 Life cycle of a Q-learning reinforcement learning algorithm
The Q-learning life-cycle figure helps visualize how a trial fits into repeated training. section 10.6
