liveBook · Manning

Ordinary least squares

Also called OLS

Ordinary least squares is a method for estimating model parameters by minimizing the sum of squared differences between observed values and model predictions.

How 5 Manning books explain it

Geometry for Programmers

Least squares is a technique that chooses model parameters by minimizing the sum of squared discrepancies between predicted and observed values.

How it is used
Uses polynomial fitting to turn data points into interpretable curves and trends.
How deep it goes
Develops the error metric, generalized polynomial formulation, matrix construction, and SymPy solution, but defers elaborate derivations.
Examples
Uses rowing-time data, polynomial degrees from zero upward, plotted trends, and repeated endpoint observations.
What you can do after
Build it from scratch with Python and SymPy by assembling and solving the least-squares coefficient system.

Read how Geometry for Programmers explains ordinary least squares →

Effective Data Analysis

An ordinary least squares regression model estimates coefficients by minimizing the sum of squared differences between observed and predicted outcomes.

How it is used
Uses linear predictors to estimate outcomes and evaluate model behavior through coefficients, accuracy metrics, and residuals.
How deep it goes
Explains fitted equations, coefficient roles, predictive accuracy, residuals, and diagnostic plots without developing estimation internals.
Examples
Demonstrates housing-price prediction using square footage, bedrooms, fitted values, residual scatterplots, and residual histograms.
What you can do after
Apply it via statsmodels to fit a regression, generate predictions, inspect coefficients, and diagnose residual behavior.

Read how Effective Data Analysis explains ordinary least squares →

Ensemble Methods for Machine Learning

Ordinary least squares is a linear-model fitting method that minimizes the sum of squared residuals.

How it is used
Fits linear models for continuous-valued targets using squared loss without regularization.
How deep it goes
Defines the unregularized squared-loss objective and contrasts penalties, but does not derive the fitting solution.
Examples
Uses a noisy one-feature function, correlated features, scikit-learn estimators, held-out predictions, and MSE or MAD comparisons.
What you can do after
Apply OLS via scikit-learn, evaluate held-out predictions, and recognize when regularization may improve robustness.

Read how Ensemble Methods for Machine Learning explains ordinary least squares →

Grokking AI Algorithms, Second Edition

The least-squares method fits a model by minimizing the sum of squared differences between observed and predicted values.

How it is used
Fits a straight regression line to data points for estimating a dependent variable from an independent variable.
How deep it goes
Explains means, deviations, slope, intercept, and implementation steps, without pursuing theoretical proofs.
Examples
Uses carat-and-price data, scatter plots, a subway-route analogy, tabulated calculations, and Python-like pseudocode.
What you can do after
Builds a regression line manually or with a short function that returns its intercept, slope, and endpoints.

Read how Grokking AI Algorithms, Second Edition explains ordinary least squares →

Timeless Algorithms, The Foundational Ideas

Ordinary least squares estimates regression parameters by minimizing the sum of squared differences between observed and predicted responses.

How it is used
Estimates regression coefficients for describing variation, predicting responses, and assessing predictors.
How deep it goes
Derives the squared-residual objective and its equivalence to maximum likelihood under independent normal errors with constant variance.
Examples
Uses a fitted regression line and vertical distances from observations to predictions, then models responses around the line with normal errors.
What you can do after
Understand why minimizing squared residuals yields the same coefficients as maximum likelihood under normally distributed errors.

Read how Timeless Algorithms, The Foundational Ideas explains ordinary least squares →

Also covered in

What to know first

Related concepts

Questions about ordinary least squares

What is ordinary least squares?

Ordinary least squares is a method for estimating model parameters by minimizing the sum of squared differences between observed values and model predictions.

Which Manning books explain ordinary least squares?

Geometry for Programmers, Effective Data Analysis, Ensemble Methods for Machine Learning, Grokking AI Algorithms, Second Edition and Timeless Algorithms, The Foundational Ideas.

What should I know before learning ordinary least squares?

Manning books that teach ordinary least squares usually introduce squared error, normal distribution, array and parameter first.

What is ordinary least squares a kind of?

Ordinary least squares is a kind of linear regression. Linear regression is a statistical method that models a numeric response as a linear function of one or more predictor variables with fitted parameters.

What is ordinary least squares also called?

Ordinary least squares is also called OLS.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.