Ordinary least squares
Ordinary least squares is a regression parameter-estimation method that minimizes the sum of squared residuals between observed and predicted responses.
Geometry for Programmers
Least squares is a technique that chooses model parameters by minimizing the sum of squared discrepancies between predicted and observed values.
- How it is used
- Uses polynomial fitting to turn data points into interpretable curves and trends.
- How deep it goes
- Develops the error metric, generalized polynomial formulation, matrix construction, and SymPy solution, but defers elaborate derivations.
- Examples
- Uses rowing-time data, polynomial degrees from zero upward, plotted trends, and repeated endpoint observations.
- What you can do after
- Build it from scratch with Python and SymPy by assembling and solving the least-squares coefficient system.
Read how Geometry for Programmers explains ordinary least squares →
Effective Data Analysis
An ordinary least squares regression model estimates coefficients by minimizing the sum of squared differences between observed and predicted outcomes.
- How it is used
- Uses linear predictors to estimate outcomes and evaluate model behavior through coefficients, accuracy metrics, and residuals.
- How deep it goes
- Explains fitted equations, coefficient roles, predictive accuracy, residuals, and diagnostic plots without developing estimation internals.
- Examples
- Demonstrates housing-price prediction using square footage, bedrooms, fitted values, residual scatterplots, and residual histograms.
- What you can do after
- Apply it via statsmodels to fit a regression, generate predictions, inspect coefficients, and diagnose residual behavior.
Read how Effective Data Analysis explains ordinary least squares →
Ensemble Methods for Machine Learning
Ordinary least squares is a linear-model fitting method that minimizes the sum of squared residuals.
- How it is used
- Fits linear models for continuous-valued targets using squared loss without regularization.
- How deep it goes
- Defines the unregularized squared-loss objective and contrasts penalties, but does not derive the fitting solution.
- Examples
- Uses a noisy one-feature function, correlated features, scikit-learn estimators, held-out predictions, and MSE or MAD comparisons.
- What you can do after
- Apply OLS via scikit-learn, evaluate held-out predictions, and recognize when regularization may improve robustness.
Read how Ensemble Methods for Machine Learning explains ordinary least squares →
Grokking AI Algorithms, Second Edition
The least-squares method fits a model by minimizing the sum of squared differences between observed and predicted values.
- How it is used
- Fits a straight regression line to data points for estimating a dependent variable from an independent variable.
- How deep it goes
- Explains means, deviations, slope, intercept, and implementation steps, without pursuing theoretical proofs.
- Examples
- Uses carat-and-price data, scatter plots, a subway-route analogy, tabulated calculations, and Python-like pseudocode.
- What you can do after
- Builds a regression line manually or with a short function that returns its intercept, slope, and endpoints.
Read how Grokking AI Algorithms, Second Edition explains ordinary least squares →
Related concepts
- squared lossaddresses · Squared loss is the square of the difference between a predicted or fitted value and its target or observation, often summed across data points.
- maximum likelihood estimationequivalent to · Maximum likelihood estimation is a statistical method that selects parameter values maximizing the likelihood of observed data under a specified probabilistic model.
- alternating least squaresused by · Alternating least squares is an iterative optimization algorithm that alternately estimates each factor matrix while holding the others fixed.
- polynomial functionuses · A polynomial function is a function expressible as a finite sum of constant multiples of nonnegative integer powers of one or more variables.
- linear regressionkind of · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.