2 Fully Connected Networks

 

This chapter covers:

  • Implementing a training loop in PyTorch
  • Changing loss functions for regression and classification problems
  • Implementing and training a fully connected network
  • Training faster using smaller batches of data

Now that we know how PyTorch gives us Tensors to represent our data and parameters, we can progress to building our first neural networks. This starts with showing how learning happens inside PyTorch. As we described in the last chapter learning is based on the principle of optimization: we can compute a loss for how well we are doing and use gradients to minimize that loss. This is how neural networks the parameters of a network are “learned” from the data, and is also the basis of many different machine learning algorithms. For these reasons optimization of loss functions is the foundation PyTorch is built from. So to implement any kind of neural network in PyTorch we must phrase the problem as an optimization problem (remember that this is also called function minimization).

2.1 Neural Networks as Optimization

2.1.1 Linear Regression

2.2 Building Our First Neural Network

2.2.1 Adding Non-Linearities

2.3 Classification Problems

2.3.1 Classification Toy Problem

2.3.2 Classification Loss Function

2.4 Better Training Code

2.4.1 Custom Metrics

2.4.2 Training and Testing Passes

2.4.3 Saving Checkpoints

2.4.4 Putting It All Together

2.5 Training in Batches

2.6 Exercises

2.7 Chapter summary