liveBook · Manning

Linear regression

As explained in Machine Learning System Design

A linear model predicts an output as a linear combination of input features, optionally followed by a task-specific link or threshold function.

Suppose you need to forecast grocery sales. Before investing in a specialized forecasting system, try a linear regression baseline: predict sales from a linear combination of available features, then ask whether its improvement over a simpler rule justifies the extra work. A baseline is useful even when it is not the final model—it shows how much the data and features can already explain. section 8.3 section 8.6.1

Build and judge the baseline in stages:

  • Start with a constant or rule-based prediction. For weekly grocery sales, one simple prediction is the same item’s sales one week earlier. section 8.3 section 8.6.1
  • Fit a linear regression using basic features. Candidate inputs include sales at several lags and rolling sums, minima, maxima, averages, medians, or quantiles; price and customer counts may add useful information. Quantile loss is an option when the forecast targets quantiles. section 8.6.1
  • Compare the gain in the chosen metric with implementation, training, debugging, testing, and computation costs. More complexity is worthwhile only when the improvement matters enough. section 8.5

A good metric is not the only check. Simple linear regression assumes a linear predictor–target relationship, zero-centered residuals, normally distributed residuals, constant residual variance, and independent errors. After fitting, inspect residuals for patterns, bias, changing variance, and highly influential observations. A residuals-versus-fitted plot helps reveal departures from linearity; a normal Q-Q plot helps check the residual distribution. Unexpected patterns are reasons to revisit the features or consider another model. section 9.2.2

Figure 9.11 Four diagnostic plots for linear regression’s residual analysis in both cases (Case 2: Assumptions are not met)
figure

Use the diagnostic-plot example to see what failed regression assumptions can look like, rather than relying on the forecast metric alone. section 9.2.2

Keep the task distinction clear: a simple linear model can also classify images using representations from a pretrained model, but that is a classification baseline, not the grocery-sales regression example. In either setting, a simple model can be hard to beat. section 8.4 section 8.3

How Machine Learning System Design covers it

How deep it goes
Works through feature choices, comparison criteria, and residual checks without deriving a fitting algorithm.
Examples
Uses grocery-sales forecasts built from lagged and rolling values, plus residual-versus-fitted and Q-Q diagnostic plots.
What it assumes
Assumes familiarity with features, lagged observations, and residuals rather than teaching regression fitting from zero.
What you can do after
Diagnose/debug with it by checking residual plots and reconsidering features or model choice when assumptions fail.
How it connects
Connects regression to baseline models, feature engineering, quantile loss, residual analysis, and time-series forecasting.
Cover of Machine Learning System Design
liveBookmanning.com
Read Machine Learning System Design on liveBook Get the book at manning.com

Also explained in

Compare how 22 books explain linear regression →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.