Residual error
A residual is the difference between an observed value and the corresponding value predicted or fitted by a model.
Statistics Every Programmer Needs
Residual error is the difference between an observed response and its value predicted by a fitted model.
- How it is used
- Measures prediction discrepancies and supplies the errors used to evaluate linear-regression fit and assumptions.
- How deep it goes
- Works from observed-minus-predicted differences through squared-error calculation and plotted and statistical diagnostics, without developing proofs.
- Examples
- Calculates a runner’s squared prediction error and plots fitted values against residuals in Python.
- What you can do after
- Diagnose/debug with residual plots and tests, and calculate SSE from a fitted regression’s residuals.
Read how Statistics Every Programmer Needs explains residual error →
Effective Data Analysis
A residual is the difference between an observed outcome and the value predicted by a fitted model.
- How it is used
- Uses residuals to diagnose discrepancies and guide iterative regression model refinement.
- How deep it goes
- Explains residual plots, distributions, outlier investigation, and possible responses to violated assumptions.
- Examples
- Uses housing-price plots and a rat-sightings model with missing weather data to illustrate residual checks.
- What you can do after
- Apply residual diagnostics using statsmodels to inspect predictions and investigate unusual records.
Machine Learning System Design
The difference between a model prediction and its corresponding true value, especially in regression and more broadly as a sample-wise prediction error.
- How it is used
- Uses sample-wise errors to debug models and identify directions for system improvement beyond aggregate metrics.
- How deep it goes
- Defines signed residuals and pseudo-residuals, then develops diagnostic checks, assumption tests, and extreme-case analysis.
- Examples
- Illustrates residuals with search rankings, segmentation masks, and marketplace demand forecasts affected by promotions and sparse sales.
- What you can do after
- Diagnose model errors by examining residual signs, distributions, cohorts, and extreme cases.
Read how Machine Learning System Design explains residual error →
Ensemble Methods for Machine Learning
A residual is the discrepancy between an observed target and a model prediction, or the corresponding per-example loss gradient in gradient boosting.
- How it is used
- Directs each gradient-boosting stage toward examples whose current predictions remain inaccurate.
- How deep it goes
- Defines residuals, derives their squared-loss gradient relation, and walks through iterative tree fitting and step-length selection.
- Examples
- Uses noisy two-moons classification and linear-regression under- and overprediction diagrams to illustrate residuals.
- What you can do after
- Build it from scratch by computing residuals, fitting weak regressors, selecting step lengths, and updating an additive ensemble.
Read how Ensemble Methods for Machine Learning explains residual error →
Grokking Machine Learning
A residual is the difference between an observed target label and a model's predicted target value.
- How it is used
- Uses label-minus-prediction errors to train successive weak learners in gradient boosting and XGBoost.
- How deep it goes
- Defines residuals as label-minus-prediction differences and works through their recalculation after successive tree predictions.
- Examples
- Calculates residuals for app-engagement labels, with a prediction of 4 and label of 7 yielding a residual of 3.
- What you can do after
- Apply residuals by subtracting predictions from labels and fitting successive trees to the remaining errors.
Read how Grokking Machine Learning explains residual error →
Experimentation for Engineers
A residual is the portion of an observed value that remains unexplained after subtracting a model estimate.
- How it is used
- Uses residuals to refine a mean baseline when estimating CPU time or other metric functions.
- How deep it goes
- Defines residuals and works through weighted residual estimation, while deferring broader validation details such as LOOCV.
- Examples
- Uses CPU-time measurements, a mean baseline, kernel weights, and numerical Gaussian process estimates.
- What you can do after
- Apply residual modeling in Python to produce weighted estimates from measured values.
Read how Experimentation for Engineers explains residual error →
Statistics Slam Dunk
A residual is the difference between an observed response value and its fitted value in a statistical model.
- How it is used
- Uses observed–fitted differences to assess regression and ANOVA model assumptions and goodness of fit.
- How deep it goes
- Defines residuals and interprets diagnostic plots for normality, linearity, and equal variance without deriving model internals.
- Examples
- Demonstrates NBA wins, group means, Residuals vs. Fitted plots, Normal QQ plots, and augmented R model output.
- What you can do after
- Diagnose/debug with base R plots and augmented model output to identify assumption problems and inspect prediction errors.
Grokking Statistics
A residual is the difference between an observed response value and the value predicted by a fitted model.
- How it is used
- Uses residuals to measure regression fit and support model evaluation.
- How deep it goes
- Defines residuals, explains their signs, squares them into loss, and stops short of detailed residual diagnostics.
- Examples
- Uses temperature and sports drink sales, plotted data points, fitted lines, and Python sum-of-squares calculations.
- What you can do after
- Diagnose regression fit by calculating residuals, summing their squares, and interpreting the result with visual checks.
Causal Inference for Data Science
A residual is the difference between an observed value and its value predicted by a fitted model.
- How it is used
- Uses prediction errors to remove confounder influence before estimating a causal effect.
- How deep it goes
- Defines residuals, gives the residualization steps, derives the linear intuition, and demonstrates R and Python implementations.
- Examples
- Demonstrates airline pricing and synthetic treatment-outcome data analyzed with linear regressions.
- What you can do after
- Apply it via linear regression to estimate treatment effects after controlling for confounders.
Read how Causal Inference for Data Science explains residual error →
Time Series Forecasting Using Foundation Models
A forecast residual is the difference between an actual value and its predicted value.
- How it is used
- Measures forecast discrepancy and supplies scores for time-series prediction intervals.
- How deep it goes
- Defines the difference and demonstrates its use in cross-validation and N-BEATS residual connections.
- Examples
- Uses two cross-validation windows with a three-step horizon and absolute actual–predicted differences.
- What you can do after
- Apply it via cross-validation to calculate conformity scores for conformal prediction intervals.
Read how Time Series Forecasting Using Foundation Models explains residual error →
Outlier Detection in Python
A time-series residual is the portion of an observation remaining after modeled trend and seasonal components are removed.
- How it is used
- Uses decomposed residuals to assess whether time-series observations remain unusual after trend and seasonality are removed.
- How deep it goes
- Explains component subtraction and demonstrates decomposition, including period selection and LOESS trend smoothing.
- Examples
- Uses regular ten-second spikes, a synthetic sine cycle, an upward trend, noise, and a 365-day decomposition.
- What you can do after
- Diagnose potential time-series outliers with statsmodels seasonal_decompose and inspect residual, trend, and seasonal components.
Read how Outlier Detection in Python explains residual error →
Also covered in
- Timeless Algorithms, The Foundational IdeasIn early access (MEAP)
- Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
- Build AI Drug Discovery PipelinesIn early access (MEAP)
- Machine Learning Algorithms in Depth
What to know first
- normal distributionA normal distribution is a continuous probability distribution with a symmetric, bell-shaped density determined by its mean and variance.
- linear regressionLinear regression is a statistical method that models a numeric response as a linear function of one or more predictor variables with fitted parameters.
- AdaBoostAdaBoost is an ensemble-learning algorithm that sequentially combines weak learners, adaptively reweighting training examples so that later learners emphasize those earlier learners handled poorly.
- gradient descentGradient descent is an iterative optimization method that updates parameters in the direction opposite an objective function’s gradient to reduce the objective’s value.
Related concepts
- gradient boostingused by · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models fitted to reduce a loss by following its gradient.
- conformity scoreused by · A conformity score is a numerical measure of how atypical an observation is relative to a prediction, here computed as the absolute forecast residual.
- FWL strategyused by · The FWL strategy estimates a regression coefficient by removing confounder-predicted components from both the treatment and outcome and regressing the resulting residuals on each other.
- negative gradientequivalent to · A negative gradient is the vector opposite a function’s gradient, pointing in the direction of steepest local decrease.
Questions about residual error
What is residual error?
A residual is the difference between an observed value and the corresponding value predicted or fitted by a model.
Which Manning books explain residual error?
Statistics Every Programmer Needs, Effective Data Analysis, Machine Learning System Design, Ensemble Methods for Machine Learning, Grokking Machine Learning, Experimentation for Engineers, Statistics Slam Dunk, Grokking Statistics and 3 more.
What should I know before learning residual error?
Manning books that teach residual error usually introduce normal distribution, linear regression, AdaBoost and gradient descent first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.










