This chapter covers:
- Understanding how algorithms can learn from data
- Reframing learning as parameter estimation, using differentiation and gradient descent
- Walking through a very simple learning algorithm from scratch
- How PyTorch supports learning with autograd
With the blooming of machine learning that has occurred over the last decade, the notion of machines that learn from experience has become a mainstream theme in both technical and journalistic circles. Now, how is it exactly that a machine learns? What are the mechanics of it, or, in other words, the algorithm behind it? From the point of view of an outer observer, a learning algorithm is presented input data that is paired with desired outputs. Once learning has occurred, that algorithm will be capable of producing correct outputs when it is fed new data that is similar enough to the input data it was trained on. With deep learning, this process works even when the input data and the desired output are far from each other, when they come from different domains, like an image and a sentence describing it, as we’ve seen in Chapter 2.