Chapter 1. What is reinforcement learning?

This chapter covers

A brief review of machine learning
Introducing reinforcement learning as a subfield
The basic framework of reinforcement learning

Computer languages of the future will be more concerned with goals and less with procedures specified by the programmer.

Marvin Minksy, 1970 ACM Turing Lecture

If you’re reading this book, you are probably familiar with how deep neural networks are used for things like image classification or prediction (and if not, just keep reading; we also have a crash course in deep learning in the appendix). Deep reinforcement learning (DRL) is a subfield of machine learning that utilizes deep learning models (i.e., neural networks) in reinforcement learning (RL) tasks (to be defined in section 1.2). In image classification we have a bunch of images that correspond to a set of discrete categories, such as images of different kinds of animals, and we want a machine learning model to interpret an image and classify the kind of animal in the image, as in figure 1.1.

Figure 1.1. An image classifier is a function or learning algorithm that takes in an image and returns a class label, classifying the image into one of a finite number of possible categories or classes.

1.1. The “deep” in deep reinforcement learning

Chapter 1. What is reinforcement learning?

This chapter covers

Figure 1.1. An image classifier is a function or learning algorithm that takes in an image and returns a class label, classifying the image into one of a finite number of possible categories or classes.

1.1. The “deep” in deep reinforcement learning

1.2. Reinforcement learning

1.3. Dynamic programming versus Monte Carlo

1.4. The reinforcement learning framework

1.5. What can I do with reinforcement learning?

1.6. Why deep reinforcement learning?

1.7. Our didactic tool: String diagrams

1.8. What’s next?

Summary