concept discount factor in category reinforcement learning

This is an excerpt from Manning's book Grokking Deep Reinforcement Learning MEAP V14 epub. Login to get full access to this book.

This number is called the discount factor, or gamma. The discount factor adjusts the importance of rewards over time. The later we receive rewards, the less attractive they are to present calculations. Another important reason why the discount factor is commonly used is to reduce the variance of return estimates. Given that the future is uncertain, and that the later we look into the future, the more stochasticity we accumulate and the more variance our value estimates will have, the discount factor helps reducing the degree to which future rewards affect our value function estimates which stabilizes learning for most agents.

Figure 2.18 Effect of discount factor and time on the value of rewards