liveBook · Manning

Linear regression

As explained in Causal Inference for Data Science

Linear regression is a regression method that estimates a linear relationship between an outcome and one or more predictors.

Linear regression estimates how an outcome changes with a predictor, but a coefficient answers a causal question only when the regression matches the causal graph. The practical decision is not simply whether to include Z: it is whether the desired quantity is a direct effect or a total effect, and whether Z is a confounder, mediator, or collider. section 6.1.2

To control for Z means including it as a covariate in a regression of Y on X. With a confounder, Y ~ X + Z adjusts the association needed to estimate both the direct and total effect. With a mediator, Y ~ X + Z estimates the direct effect, while Y ~ X estimates the total effect, including the path through Z. With a collider, conditioning on Z is not required for either effect. section 6.1.2

Table 6.8 Effects between X and Y, and how to estimate them
Case
Direct effect
Estimation direct effect
Total effect
Estimation total effect
Confounder
a
Y ~ X + Z
a
Y ~ X + Z
Mediator
a
Y ~ X + Z
a + bc
Y ~ X
Collider
a
Y ~ X
a
Y ~ X

Compare the coefficient targets and regression formulas across the three causal roles of Z before treating a fitted coefficient as an effect. section 6.1.2

The same logic connects regression to intervention. For a continuous treatment X, the effect of increasing X by one unit is expressed with the do operator. In linear models, this average treatment effect is constant across starting values of X, producing a global effect rather than a separate comparison for every possible value. section 6.2.4

Regression can also reveal why an apparently plausible estimate fails. When treatment and outcome share an unobserved confounder, the coefficient from Y ~ D is biased. An instrumental-variable strategy instead regresses D on Z and Y on Z, then uses the ratio of their coefficients to estimate the effect of D on Y, provided the stated instrument relationships hold. section 9.2.1

A biased treatment coefficient

In a synthetic example with a true treatment effect of −2, a naive regression of outcome on treatment returns about 1.0005 because of an unobserved confounder. An IV estimate returns about −1.8898; R code also fits the IV model with AER using outcome ~ treatment | instrument. section 9.3

Linear regression is therefore not evidence of causation by itself. Its usefulness depends on identifying the causal paths, choosing covariates for the estimand, and recognizing when unobserved confounding requires another design such as instrumental variables. It also has practical limitations when used for regression discontinuity designs. section 6.1.2 section 11.5

How Causal Inference for Data Science covers it

How deep it goes
Works through covariate specifications, do-operator effects, and instrumental-variable coefficient ratios; flags RDD limitations without detailing them.
Examples
Uses treatment, recovery, side effects, synthetic confounding data, and R and Python regression or IV code.
What it assumes
Builds from causal paths and adjustment, then assumes familiarity with treatment effects, expectations, and regression coefficients.
What you can do after
Evaluate/understand which regression specification targets a direct or total effect and when a naive coefficient is biased.
How it connects
Connects linear regression to causal diagrams, confounders, mediators, colliders, the do operator, adjustment, and instrumental variables.
Cover of Causal Inference for Data Science
liveBookmanning.com
Read Causal Inference for Data Science on liveBook Get the book at manning.com

Also explained in

Compare how 22 books explain linear regression →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Causal Inference for Data Science. The section links point to the parts of the book it draws on.