Credit assignment asks which earlier decisions contributed to a later success or failure. In reinforcement learning, this is difficult when rewards arrive after a sequence of actions: the agent must learn from consequences rather than labeled correct answers. section 1.1 section 6.7
Eligibility traces address delayed credit by letting a temporal-difference error reach recently visited state-action pairs:
- TD(λ) uses short-term memory to propagate the error backward; λ tunes how far back credit reaches, from one-step TD at 0 to Monte Carlo at 1. section 6.7
- SARSA(λ) combines eligibility traces with on-policy learning. Watkins’s Q(λ) resets traces after a non-greedy exploratory action. section 6.7
Choosing a path
An agent tries paths that yield rewards of 100, 200, 30, or 50. It must explore and learn which decisions lead to higher cumulative outcomes. section 1.1
Figure 1.1 Reinforcement learning in the context of machine learning.
Place reward-driven action learning in context alongside supervised learning. section 1.1

