liveBook · Manning

Residual error

A residual is the difference between an observed value and the corresponding value predicted or fitted by a model.

How 11 Manning books explain it

Statistics Every Programmer Needs

Residual error is the difference between an observed response and its value predicted by a fitted model.

How it is used
Measures prediction discrepancies and supplies the errors used to evaluate linear-regression fit and assumptions.
How deep it goes
Works from observed-minus-predicted differences through squared-error calculation and plotted and statistical diagnostics, without developing proofs.
Examples
Calculates a runner’s squared prediction error and plots fitted values against residuals in Python.
What you can do after
Diagnose/debug with residual plots and tests, and calculate SSE from a fitted regression’s residuals.

Read how Statistics Every Programmer Needs explains residual error →

Effective Data Analysis

A residual is the difference between an observed outcome and the value predicted by a fitted model.

How it is used
Uses residuals to diagnose discrepancies and guide iterative regression model refinement.
How deep it goes
Explains residual plots, distributions, outlier investigation, and possible responses to violated assumptions.
Examples
Uses housing-price plots and a rat-sightings model with missing weather data to illustrate residual checks.
What you can do after
Apply residual diagnostics using statsmodels to inspect predictions and investigate unusual records.

Read how Effective Data Analysis explains residual error →

Machine Learning System Design

The difference between a model prediction and its corresponding true value, especially in regression and more broadly as a sample-wise prediction error.

How it is used
Uses sample-wise errors to debug models and identify directions for system improvement beyond aggregate metrics.
How deep it goes
Defines signed residuals and pseudo-residuals, then develops diagnostic checks, assumption tests, and extreme-case analysis.
Examples
Illustrates residuals with search rankings, segmentation masks, and marketplace demand forecasts affected by promotions and sparse sales.
What you can do after
Diagnose model errors by examining residual signs, distributions, cohorts, and extreme cases.

Read how Machine Learning System Design explains residual error →

Ensemble Methods for Machine Learning

A residual is the discrepancy between an observed target and a model prediction, or the corresponding per-example loss gradient in gradient boosting.

How it is used
Directs each gradient-boosting stage toward examples whose current predictions remain inaccurate.
How deep it goes
Defines residuals, derives their squared-loss gradient relation, and walks through iterative tree fitting and step-length selection.
Examples
Uses noisy two-moons classification and linear-regression under- and overprediction diagrams to illustrate residuals.
What you can do after
Build it from scratch by computing residuals, fitting weak regressors, selecting step lengths, and updating an additive ensemble.

Read how Ensemble Methods for Machine Learning explains residual error →

Grokking Machine Learning

A residual is the difference between an observed target label and a model's predicted target value.

How it is used
Uses label-minus-prediction errors to train successive weak learners in gradient boosting and XGBoost.
How deep it goes
Defines residuals as label-minus-prediction differences and works through their recalculation after successive tree predictions.
Examples
Calculates residuals for app-engagement labels, with a prediction of 4 and label of 7 yielding a residual of 3.
What you can do after
Apply residuals by subtracting predictions from labels and fitting successive trees to the remaining errors.

Read how Grokking Machine Learning explains residual error →

Experimentation for Engineers

A residual is the portion of an observed value that remains unexplained after subtracting a model estimate.

How it is used
Uses residuals to refine a mean baseline when estimating CPU time or other metric functions.
How deep it goes
Defines residuals and works through weighted residual estimation, while deferring broader validation details such as LOOCV.
Examples
Uses CPU-time measurements, a mean baseline, kernel weights, and numerical Gaussian process estimates.
What you can do after
Apply residual modeling in Python to produce weighted estimates from measured values.

Read how Experimentation for Engineers explains residual error →

Statistics Slam Dunk

A residual is the difference between an observed response value and its fitted value in a statistical model.

How it is used
Uses observed–fitted differences to assess regression and ANOVA model assumptions and goodness of fit.
How deep it goes
Defines residuals and interprets diagnostic plots for normality, linearity, and equal variance without deriving model internals.
Examples
Demonstrates NBA wins, group means, Residuals vs. Fitted plots, Normal QQ plots, and augmented R model output.
What you can do after
Diagnose/debug with base R plots and augmented model output to identify assumption problems and inspect prediction errors.

Read how Statistics Slam Dunk explains residual error →

Grokking Statistics

A residual is the difference between an observed response value and the value predicted by a fitted model.

How it is used
Uses residuals to measure regression fit and support model evaluation.
How deep it goes
Defines residuals, explains their signs, squares them into loss, and stops short of detailed residual diagnostics.
Examples
Uses temperature and sports drink sales, plotted data points, fitted lines, and Python sum-of-squares calculations.
What you can do after
Diagnose regression fit by calculating residuals, summing their squares, and interpreting the result with visual checks.

Read how Grokking Statistics explains residual error →

Causal Inference for Data Science

A residual is the difference between an observed value and its value predicted by a fitted model.

How it is used
Uses prediction errors to remove confounder influence before estimating a causal effect.
How deep it goes
Defines residuals, gives the residualization steps, derives the linear intuition, and demonstrates R and Python implementations.
Examples
Demonstrates airline pricing and synthetic treatment-outcome data analyzed with linear regressions.
What you can do after
Apply it via linear regression to estimate treatment effects after controlling for confounders.

Read how Causal Inference for Data Science explains residual error →

Time Series Forecasting Using Foundation Models

A forecast residual is the difference between an actual value and its predicted value.

How it is used
Measures forecast discrepancy and supplies scores for time-series prediction intervals.
How deep it goes
Defines the difference and demonstrates its use in cross-validation and N-BEATS residual connections.
Examples
Uses two cross-validation windows with a three-step horizon and absolute actual–predicted differences.
What you can do after
Apply it via cross-validation to calculate conformity scores for conformal prediction intervals.

Read how Time Series Forecasting Using Foundation Models explains residual error →

Outlier Detection in Python

A time-series residual is the portion of an observation remaining after modeled trend and seasonal components are removed.

How it is used
Uses decomposed residuals to assess whether time-series observations remain unusual after trend and seasonality are removed.
How deep it goes
Explains component subtraction and demonstrates decomposition, including period selection and LOESS trend smoothing.
Examples
Uses regular ten-second spikes, a synthetic sine cycle, an upward trend, noise, and a 365-day decomposition.
What you can do after
Diagnose potential time-series outliers with statsmodels seasonal_decompose and inspect residual, trend, and seasonal components.

Read how Outlier Detection in Python explains residual error →

Also covered in

What to know first

Related concepts

Questions about residual error

What is residual error?

A residual is the difference between an observed value and the corresponding value predicted or fitted by a model.

Which Manning books explain residual error?

Statistics Every Programmer Needs, Effective Data Analysis, Machine Learning System Design, Ensemble Methods for Machine Learning, Grokking Machine Learning, Experimentation for Engineers, Statistics Slam Dunk, Grokking Statistics and 3 more.

What should I know before learning residual error?

Manning books that teach residual error usually introduce normal distribution, linear regression, AdaBoost and gradient descent first.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.