Residual error

As explained in Causal Inference for Data Science

A residual is the difference between an observed value and its value predicted by a fitted model.

A residual is the difference between an observed value and its value predicted by a fitted model. In causal analysis, residuals retain variation that the chosen confounders cannot predict. section 8.1

A residual-based adjustment removes confounder influence in four steps:

• Predict treatment D from confounders C and calculate treatment residuals: the actual treatment minus its prediction. section 8.1.1
• Predict outcome O from C and calculate outcome residuals in the same way. section 8.1.1
• Regress the outcome residuals on the treatment residuals; the coefficient estimates the effect of D on O in the linear setup. section 8.1 section 8.1.1

Airline pricing

For an airline, ticket price D is the treatment, ticket sales O are the outcome, and seasonality and inflation C are confounders. Residualizing both price and sales against C isolates variation used to estimate the price effect. section 8.1

Figure 8.1 Steps to run the DML algorithm. The main goal of this algorithm is to first get rid of the influence of confounder C on both the decision and outcome variables. This is done using two predictive models. What remains after their predictions, called residuals, is used to estimate the causal effect from D to O.

Follow the algorithm diagram to see how two predictive models remove confounder influence before residual regression. section 8.1

How Causal Inference for Data Science covers it

How deep it goes

Defines residuals, gives the residualization steps, derives the linear intuition, and demonstrates R and Python implementations.

Examples

Demonstrates airline pricing and synthetic treatment-outcome data analyzed with linear regressions.

What it assumes

Assumes familiarity with treatments, outcomes, confounders, prediction, and linear regression.

What you can do after

Apply it via linear regression to estimate treatment effects after controlling for confounders.

How it connects

Connects residuals with confounders, regression, the FWL theorem, causal effects, and double machine learning.

liveBookmanning.com

Read Causal Inference for Data Science on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Statistics Every Programmer NeedsResidual error is the difference between an observed response and its value predicted by a fitted model.
• 

  liveBookmanning.com
  Effective Data AnalysisA residual is the difference between an observed outcome and the value predicted by a fitted model.
• 

  liveBookmanning.com
  Machine Learning System DesignThe difference between a model prediction and its corresponding true value, especially in regression and more broadly as a sample-wise prediction error.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningA residual is the discrepancy between an observed target and a model prediction, or the corresponding per-example loss gradient in gradient boosting.
• 

  liveBookmanning.com
  Grokking Machine LearningA residual is the difference between an observed target label and a model's predicted target value.
• 

  liveBookmanning.com
  Experimentation for EngineersA residual is the portion of an observed value that remains unexplained after subtracting a model estimate.
• 

  liveBookmanning.com
  Statistics Slam DunkA residual is the difference between an observed response value and its fitted value in a statistical model.
• 

  liveBookmanning.com
  Grokking StatisticsA residual is the difference between an observed response value and the value predicted by a fitted model.
• 

  liveBookmanning.com
  Time Series Forecasting Using Foundation ModelsA forecast residual is the difference between an actual value and its predicted value.
• 

  liveBookmanning.com
  Outlier Detection in PythonA time-series residual is the portion of an observation remaining after modeled trend and seasonal components are removed.

Compare how 11 books explain residual error →

Related concepts

• gradient boostingused by · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.
• sum of squarespart of · A sum of squares is a quantity obtained by squaring specified values, deviations, or residuals and adding the resulting squares.
• conformity scoreused by · A conformity score is a numerical measure of how atypical an observation is relative to a prediction, here computed as the absolute forecast residual.

This explainer was generated by Manning's knowledge graph from the text of Causal Inference for Data Science. The section links point to the parts of the book it draws on.