Linear regression predicts a continuous value by scaling an input and adding an offset. It is a useful first model when two measurements might be related by a line. section 5.2.1 section 5.2.4
Reading an unfamiliar thermometer
Suppose a thermometer has no units, but you have several readings paired with Celsius temperatures. Try `t_c = w * t_u + b`: `t_u` is the unfamiliar reading, `w` is the weight that scales it, and `b` is the bias added afterward. Fitting the model means finding values of `w` and `b` that bring its predictions close to the measured temperatures. section 5.2.1 section 5.2.4
A loss function measures prediction error. Training repeatedly updates the parameters to lower that loss. In PyTorch, gradients computed through autograd can guide those updates; the same forward-pass, loss, backward-pass, and update pattern also applies to neural networks. section 5.2.4 section 5.8
A line cannot change slope at different input values. Neural networks extend the scaling-and-offset operation with nonlinear activation functions, then compose many such units to fit more complicated relationships. Extra flexibility is not always better: on a small, noisy temperature dataset, even a tiny network can chase individual measurements rather than fit a simple line. section 6.1.3 section 6.3.3
Figure 6.6 Composing multiple linear units and tanh activation functions to produce nonlinear outputs
See how composing linear units with tanh activations produces outputs more varied than a single line. section 6.1.6
