6 Using A Neural Network To Fit Our Data

 

This chapter covers:

  • The use of non-linear activation functions as the key difference from linear models
  • The many different kinds of activation functions in common use
  • PyTorch’s nn module, containing neural network building blocks
  • Solving a simple linear-fit problem with a neural network

So far we took a close look on how a linear model can learn and how to make it happen in PyTorch. We have focused on a very simple regression problem, that only required us to use a linear model with one input and one output. Such a simple example allowed us to dissect the mechanics of a model that learns without getting overly distracted with the implementation of the model itself. As we saw in the overview diagram in Chapter 5 (repeated here as Figure 6.1) Figure-5.2, the exact details of the model are not needed to understand the high-level process that trains a model. Back-propagating error to the parameters, then updating those parameters by taking the gradient with respect to the loss is going to be the same no matter what the underlying model is.

Figure 6.1. Our mental model of the learning process, as implemented in chapter 5.
p1ch1learningprocess

In this chapter we are only going to be making changes to our model architecture. We’re going to implement a full artificial neural network to solve our problem.

6.1  Artificial Neurons

6.1.1  All We Need is Activation

6.1.2  What learning means for a neural network

6.2  The PyTorch nn module

6.2.1  Finally a Neural Network

6.3  Subclassing nn.Module

6.3.1  The Functional API

6.4  Conclusion

6.5  Exercises

6.6 Summary