concept sequential feedback in category reinforcement learning

This is an excerpt from Manning's book Grokking Deep Reinforcement Learning.
1.1.5 Deep reinforcement learning agents learn from sequential feedback
The action taken by the agent may have delayed consequences. The reward may be sparse and only manifest after several time steps. Thus the agent must be able to learn from sequential feedback. Sequential feedback gives rise to a problem referred to as the temporal credit assignment problem. The temporal credit assignment problem is the challenge of determining which state and/or action is responsible for a reward. When there is a temporal component to a problem, and actions have delayed consequences, it becomes challenging to assign credit for rewards.
Figure 1.9 The difficulty of the temporal credit assignment problem
![]()
In chapter 3, we’ll study the ins and outs of sequential feedback in isolation. That is, your programs learn from simultaneously sequential, supervised (as opposed to evaluative) and exhaustive (as opposed to sampled) feedback.
Chapter 2 is a foundation for all chapters in this book use. Chapter 3 is about planning algorithms that deal with sequential feedback. Chapter 4 is about bandit algorithms that deal with evaluative feedback. Chapters 5, 6, and 7 are about RL algorithms, algorithms that deal with feedback that is simultaneously sequential and evaluative. This type of problem is what people refer to as ‘tabular’ reinforcement learning. Starting from this chapter, we dig into the details of deep reinforcement learning.