Linear regression is a model for quantitative estimation: it maps an input vector to a predicted numeric output. With weights w and bias b, its prediction is ŷ = wᵀx + b = Σⱼwⱼxⱼ + b. Training estimates the weights and bias from observed input–output pairs by reducing prediction error. It is suitable when the underlying function is simple; more complex relationships require nonlinear models. section 3.5.1 section 1.1
The weighted-sum portion of the model is a linear transformation: a transform preserves linear combinations, so transforming a weighted combination of vectors gives the same weighted combination of their transformed versions. In finite-dimensional spaces, every matrix-vector multiplication is such a transformation. The added bias makes the full prediction formula an affine shift rather than a matrix multiplication alone. section 2.10.1 section 2.10.2 section 3.5.1
A linear regression workflow is:
- Represent each observation as an input vector xᵢ paired with an observed target yᵢ. section 3.5.1
- Compute predictions with wᵀxᵢ + b and measure squared error against each target. section 3.5.1
- Sum errors across a batch, then adjust weights and bias in the direction that reduces error. section 3.5.1
- Scale features to a common range when their values have very different magnitudes. section 8.5
Learning a one-variable prediction
A synthetic target follows y = 1.5x + 2.73 with added noise. Starting from current w and b, repeatedly predict y_pred = wx + b, compute mean squared error, calculate gradients for w and b, and update both parameters. The learned result closely approaches y_pred = 1.50059x + 2.746823, illustrating how fitting recovers a numeric relationship from noisy observations. section 3.5.1
Predicting weekly demand
An e-commerce demand predictor uses previous-week sales, discount fraction, and weeks until the next holiday as an input vector, then predicts the number of phones sold in the upcoming week. This is a regression problem because its target is a quantity rather than one of several prespecified classes. section 8.5 section 1.5
Regression and classification can be combined but solve different output problems. A regressor emits a value; thresholding that value can produce a class decision. In object detection, regressor components similarly refine bounding-box coordinates while classifier components identify object classes. section 1.5 section 11.3.1 section 11.4.3
Listing 3.1 PyTorch linear model (closed-form gradient formula needed)
x = 10 * torch.randn(N) #1
= 1.5 * x + 2.73 _obs = y + (0.5 * torch.randn(N)) #2
for step in range(num_steps):
y_pred = w*x + b #3
mean_squared_error = torch.mean(
(y_pred - y_obs) ** 2) #4
w_grad = torch.mean(2 * ((y_pred - y_obs)* x))
b_grad = torch.mean(2 * (y_pred - y_obs)) #5
w = w - learning_rate * w_grad
b = b - learning_rate * b_grad #6
print("True function: y = 1.5*x + 2.73")
print("Learned function: y_pred = {}*x + {}".format(w[0], b[0]))
This implementation places the prediction formula, mean squared error, gradients, and parameter updates side by side. section 3.5.1
Table 8.1 Sample training data for demand prediction
| Previous week sales |
Discount fraction (%) |
Weeks to next holidays |
Number of units sold |
|---|---|---|---|
| 76,440 |
63 |
2 |
94,182 |
| 41,512 |
50 |
3 |
51,531 |
| 77,395 |
77 |
9 |
95,938 |
| … |
… |
… |
… |
| 21,532 |
70 |
4 |
28,559 |
This table makes the multi-feature demand-prediction inputs and numeric target concrete. section 8.5
