concept exploration in category reinforcement learning

appears as: exploration, exploration
Grokking Deep Reinforcement Learning

This is an excerpt from Manning's book Grokking Deep Reinforcement Learning.

The reward received by the agent may be weak, in the sense that it may provide no supervision. The reward may indicate goodness and not correctness, meaning it may contain no information about other potential rewards. Thus the agent must be able to learn from evaluative feedback. Evaluative feedback gives rise to the need for exploration. The agent must be able to balance the gathering of information with the exploitation of current information. This is also referred to as the exploration vs. exploitation tradeoff.

Figure 1.10 The difficulty of the exploration vs. exploitation tradeoff
01_10
Figure 1.10 The difficulty of the exploration vs. exploitation tradeoff
01_10

Let’s also consider the opposite side of the spectrum: a strategy with exploration but no exploitation at all. This is another fundamental baseline which we can call a random strategy or a pure exploration strategy. This is simply an approach to action selection with no exploitation at all. The sole goal of the agent is to gain information.

Do you know people that when starting a new project, spend a lot of time “researching” without jumping into the water? Me too! They can take weeks just reading papers. Remember, while exploration is essential, it must be balanced well to get maximum gains.