A reward signal is feedback about how favorable an action’s outcome is. A positive value encourages an action; a penalty discourages it. Think of giving a dog a treat when it sits: the treat helps connect the action to a desirable outcome. In reinforcement learning, an agent receives this feedback as it acts in an environment and learns through trial and error. section 10.1 section 10.2
Giving a parking-lot agent feedback
Imagine a self-driving car moving one space north, south, east, or west at a time to find its owner. Its simulated environment assigns rewards or penalties after moves: reaching the goal is valuable, while collisions are discouraged. A simple implementation returns 10 for the goal, −5 for a car, and −1 for another space. The agent can practice in the simulator, using these outcomes to learn which actions help it reach the owner. section 10.5
Figure 10.8 Rewards for specific events in the environment (actions performed)
The parking-lot reward diagram links events in the environment to the feedback the agent receives. section 10.5
The values need care. If moving between empty spaces repeatedly earns rewards, the car might keep moving back and forth instead of finding its owner. Rewards that look sensible one action at a time can therefore encourage a poor overall strategy. section 10.5
Similar scores serve different purposes:
- In reinforcement learning, rewards and penalties provide feedback from actions; their cumulative effect helps the agent learn a sequence of decisions. section 10.1
- In Minimax search, a crafted heuristic scores simulated game states so the agent can favor its own advantage while accounting for an opponent’s moves. Minimax does not learn that heuristic automatically. section 3.7
- In a genetic algorithm, a fitness score measures a candidate solution; an invalid knapsack can receive a severe penalty, such as a score of zero. section 4.7
To judge training, consider the goal and the costs that matter in that environment. Two possible measures are the number of penalties across attempts and average reward per action, calculated as cumulative reward divided by the number of actions. A clear reward function and a realistic simulation make trial-and-error learning more useful. section 10.8 section 10.12
