Residual error
Residual error is the difference between an observed value and its corresponding value predicted or fitted by a model.
Statistics Every Programmer Needs
Residual error is the difference between an observed response and its value predicted by a fitted model.
- How it is used
- Measures prediction discrepancies and supplies the errors used to evaluate linear-regression fit and assumptions.
- How deep it goes
- Works from observed-minus-predicted differences through squared-error calculation and plotted and statistical diagnostics, without developing proofs.
- Examples
- Calculates a runner’s squared prediction error and plots fitted values against residuals in Python.
- What you can do after
- Diagnose/debug with residual plots and tests, and calculate SSE from a fitted regression’s residuals.
Read how Statistics Every Programmer Needs explains residual error →
Effective Data Analysis
A residual is the difference between an observed outcome and the value predicted by a fitted model.
- How it is used
- Uses residuals to inspect regression errors and guide model refinement.
- How deep it goes
- Defines observed-minus-predicted differences and works through plots, record checks, and revision without deriving a formal residual test.
- Examples
- Plots housing-price and rat-sighting residuals and investigates a rat-sighting record with missing weather entered as zero.
- What you can do after
- Diagnose/debug with it by locating unusual errors, checking their records, and reassessing residuals after revising a regression model.
Machine Learning System Design
The difference between a model prediction and its corresponding true value, especially in regression and more broadly as a sample-wise prediction error.
- How it is used
- Uses sample-wise prediction errors to reveal failures and biases hidden by an aggregate model score.
- How deep it goes
- Defines signed residuals and develops diagnostic checks for distributions, assumptions, cohorts, and extreme cases without a mathematical derivation.
- Examples
- Uses an outlier-heavy prediction thought experiment, linear regression diagnostic plots, and marketplace demand-forecasting failures.
- What you can do after
- Diagnose/debug with it by locating biased or extreme errors and choosing targeted feature, fallback, or deployment changes.
Read how Machine Learning System Design explains residual error →
Ensemble Methods for Machine Learning
A residual is the discrepancy between an observed target and a model prediction, or the corresponding per-example loss gradient in gradient boosting.
- How it is used
- Directs each gradient-boosting stage toward examples whose current predictions remain inaccurate.
- How deep it goes
- Defines residuals, derives their squared-loss gradient relation, and walks through iterative tree fitting and step-length selection.
- Examples
- Uses noisy two-moons classification and linear-regression under- and overprediction diagrams to illustrate residuals.
- What you can do after
- Build it from scratch by computing residuals, fitting weak regressors, selecting step lengths, and updating an additive ensemble.
Read how Ensemble Methods for Machine Learning explains residual error →
Grokking Machine Learning
A residual is the difference between an observed target label and a model's predicted target value.
- How it is used
- Uses label-minus-prediction errors to train successive weak learners in gradient boosting and XGBoost.
- How deep it goes
- Defines residuals as label-minus-prediction differences and works through their recalculation after successive tree predictions.
- Examples
- Calculates residuals for app-engagement labels, with a prediction of 4 and label of 7 yielding a residual of 3.
- What you can do after
- Apply residuals by subtracting predictions from labels and fitting successive trees to the remaining errors.
Read how Grokking Machine Learning explains residual error →
Experimentation for Engineers
A residual is the portion of an observed value that remains unexplained after subtracting a model estimate.
- How it is used
- Uses residuals to refine a mean baseline when estimating CPU time or other metric functions.
- How deep it goes
- Defines residuals and works through weighted residual estimation, while deferring broader validation details such as LOOCV.
- Examples
- Uses CPU-time measurements, a mean baseline, kernel weights, and numerical Gaussian process estimates.
- What you can do after
- Apply residual modeling in Python to produce weighted estimates from measured values.
Read how Experimentation for Engineers explains residual error →
Statistics Slam Dunk
A residual is the difference between an observed response value and its fitted value in a statistical model.
- How it is used
- Uses observed–fitted differences to assess regression and ANOVA model assumptions and goodness of fit.
- How deep it goes
- Defines residuals and interprets diagnostic plots for normality, linearity, and equal variance without deriving model internals.
- Examples
- Demonstrates NBA wins, group means, Residuals vs. Fitted plots, Normal QQ plots, and augmented R model output.
- What you can do after
- Diagnose/debug with base R plots and augmented model output to identify assumption problems and inspect prediction errors.
Grokking Statistics
A residual is the difference between an observed response value and the value predicted by a fitted model.
- How it is used
- Uses residuals to measure regression fit and support model evaluation.
- How deep it goes
- Defines residuals, explains their signs, squares them into loss, and stops short of detailed residual diagnostics.
- Examples
- Uses temperature and sports drink sales, plotted data points, fitted lines, and Python sum-of-squares calculations.
- What you can do after
- Diagnose regression fit by calculating residuals, summing their squares, and interpreting the result with visual checks.
Causal Inference for Data Science
A residual is the difference between an observed value and its value predicted by a fitted model.
- How it is used
- Uses prediction errors to remove confounder influence before estimating a causal effect.
- How deep it goes
- Defines residuals, gives the residualization steps, derives the linear intuition, and demonstrates R and Python implementations.
- Examples
- Demonstrates airline pricing and synthetic treatment-outcome data analyzed with linear regressions.
- What you can do after
- Apply it via linear regression to estimate treatment effects after controlling for confounders.
Read how Causal Inference for Data Science explains residual error →
Time Series Forecasting Using Foundation Models
A forecast residual is the difference between an actual value and its predicted value.
- How it is used
- Measures forecast discrepancy and supplies scores for time-series prediction intervals.
- How deep it goes
- Defines the difference and demonstrates its use in cross-validation and N-BEATS residual connections.
- Examples
- Uses two cross-validation windows with a three-step horizon and absolute actual–predicted differences.
- What you can do after
- Apply it via cross-validation to calculate conformity scores for conformal prediction intervals.
Read how Time Series Forecasting Using Foundation Models explains residual error →
Outlier Detection in Python
A time-series residual is the portion of an observation remaining after modeled trend and seasonal components are removed.
- How it is used
- Uses decomposed residuals to assess whether time-series observations remain unusual after trend and seasonality are removed.
- How deep it goes
- Explains component subtraction and demonstrates decomposition, including period selection and LOESS trend smoothing.
- Examples
- Uses regular ten-second spikes, a synthetic sine cycle, an upward trend, noise, and a 365-day decomposition.
- What you can do after
- Diagnose potential time-series outliers with statsmodels seasonal_decompose and inspect residual, trend, and seasonal components.
Read how Outlier Detection in Python explains residual error →
Related concepts
- gradient boostingused by · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.
- sum of squarespart of · A sum of squares is a quantity obtained by squaring specified values, deviations, or residuals and adding the resulting squares.
- conformity scoreused by · A conformity score is a numerical measure of how atypical an observation is relative to a prediction, here computed as the absolute forecast residual.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.