liveBook · Manning

Multicollinearity

As explained in Causal Inference for Data Science

Multicollinearity is a regression condition in which predictors are highly correlated, making their individual effects difficult to distinguish and estimate precisely.

Multicollinearity occurs when regression predictors are highly correlated, making it difficult to distinguish their individual effects. Coefficient standard errors rise, and identical predictors produce infinitely many coefficient solutions. section 7.4.1

Consider estimating Y from treatment predictor X while adding Z. If X is nearly a copy of Z, the regression cannot reliably decide how much effect belongs to either variable. Adding Z can therefore increase uncertainty rather than improve estimation. section 7.4.1

A synthetic regression experiment

Generate Z, define X as Z plus small noise, and define Y as 3X plus noise. Regressing Y on X gives X a standard deviation of 0.08915; adding Z raises it to 0.41669, while estimated coefficients become unstable. section 7.4.1

Table 7.13 These controls are neutral in terms of bias, but not variance. The outcome predictor generally decreases the variance of your estimations, whereas the treatment predictor may increase them.
Outcome predictor
sidebar figure
Treatment predictor
sidebar figure

Compare outcome-predictor and treatment-predictor controls, which can have opposite effects on estimation variance. section 7.4.1

How Causal Inference for Data Science covers it

How deep it goes
Explains correlation, coefficient uncertainty, and nonunique solutions, then demonstrates the effect with simulated regressions.
Examples
Uses synthetic X, Z, and Y data with R and Python regressions comparing models with and without Z.
What it assumes
Assumes familiarity with regression predictors, coefficients, and standard deviations.
What you can do after
Diagnose how adding a correlated predictor changes coefficient precision by comparing regression standard errors.
How it connects
Links regression, covariate adjustment, standard error, bias, and variance.
Cover of Causal Inference for Data Science
liveBookmanning.com
Read Causal Inference for Data Science on liveBook Get the book at manning.com

Also explained in

Compare how 7 books explain multicollinearity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Causal Inference for Data Science. The section links point to the parts of the book it draws on.