Machine learning fits a model to examples so it can produce useful outputs for new, sufficiently similar inputs without receiving explicit task rules. The model contains parameters whose values are estimated from data. section 5.0 section 5.1
A typical learning process is:
- Provide input data paired with desired outputs. section 5.0
- Use a loss function to measure prediction error. section 5.8
- Compute how the loss changes with each parameter and update parameters toward lower loss. section 5.2 section 5.8
- Repeat the process, then evaluate on held-out validation data to check generalization. section 5.2 section 5.8
For neural-network training, the practical loop is forward propagation, loss calculation, backpropagation, and an optimizer step. PyTorch supplies autograd for gradients and optimizers for parameter updates; clearing accumulated gradients prevents them from affecting later updates. section 5.8
Fitting a noisy dataset
Start with a simple model and noisy observations, estimate its parameters by hand, and progressively let PyTorch perform differentiation and optimization. The same exercise exposes the essential mechanics later used to train neural networks, even though the initial model is not itself a neural network. section 5.2 section 5.6
Figure 5.2 Our mental model of the learning process.
Use the mental model of learning to connect examples, model predictions, error, gradients, and parameter updates into one iterative process. section 5.2
