Reinforcement learning trains an agent to act in an environment using feedback from its actions. Useful actions earn rewards; harmful actions receive penalties. Like a dog learning to sit after repeated treats, the agent tries actions, observes what happens, and learns which behavior helps it reach a goal. section 10.1 section 10.2
Choose this approach when the goal and available actions are known, but the best sequence of actions is not. Unlike supervised learning, it does not learn from labeled answers to each question; unlike unsupervised learning, it is not chiefly looking for clusters or trends. It learns from the consequences of acting. section 10.1 section 10.3
A car in a parking lot
Imagine a simulated parking lot containing a car, other vehicles, pedestrians, and the car’s owner. The agent can move north, south, east, or west. Its goal is to reach the owner while avoiding collisions. A collision with a pedestrian has a more severe consequence than one with a car. Training therefore needs to assess a series of moves, not just whether each move looks good in isolation: a move with little immediate benefit may help the car reach its destination safely. section 10.3 section 10.4
The training loop can be understood in four steps:
- Set up an environment that simulates the problem and a goal that defines useful outcomes. section 10.4
- Let the agent take an available action in its current situation, or state. section 10.4
- Measure the outcome and give feedback as a reward or penalty. section 10.2
- Repeat so the agent learns which actions support the greatest cumulative reward toward its goal. section 10.1 section 10.3
Figure 10.6 The Markov Decision Process for reinforcement learning
The decision-process diagram helps connect the agent’s situation, action, and measured outcome to the repeated training cycle. section 10.4
Reinforcement learning is especially useful when decisions build toward a longer-term result. Applications include strategic games, robotics, and industrial-process automation. In games too complex to solve by simply searching for a winning move, agents can improve through many rounds of play and feedback. The approach is most practical when a realistic environment can be simulated at scale and a clear reward function can be defined. section 1.31 section 10.12 section 10.3
The learning method may use classical techniques or deep learning with artificial neural networks. Reinforcement learning with human feedback also uses human preferences to help fine-tune large language models toward more helpful responses. section 10.1 section 11.37
