10 Interpretable Reinforcement Learning: Attention and Relational Models
In this chapter we learn how to:
- Implement a relational reinforcement algorithm using the popular self-attention model
- Visualize attention maps in order to better interpret the reasoning of an RL agent
- Reason about model invariance and equivariance
- Incorporate Double Q-learning to improve stability of training
Hopefully by this point you have come to appreciate just how powerful the combination of deep learning and reinforcement learning is at solving tasks previously thought to be the exclusive domain of humans. Deep learning is a class of powerful learning algorithms that can comprehend and reason through complex patterns and data, and reinforcement learning is the framework we use to solve control problems.
Throughout this book we’ve focused on using games as a laboratory for experimenting with reinforcement learning algorithms as they allow us to assess the ability of these algorithms in a very controlled setting. When we build an RL agent that learns to play a game well, we are generally satisfied our algorithm is working. Of course, reinforcement learning has many more applications outside of playing games, and in some of these other domains, the raw performance of the algorithm using some metric (e.g. the accuracy percentage on some task) is not useful without knowing how the algorithm is making its decision.