Simpson's paradox
Simpson's paradox is the reversal or substantial alteration of an association between variables when data are aggregated rather than examined within relevant subgroups.
Causal Inference for Data Science
Simpson’s paradox is a statistical phenomenon in which an association or trend observed across aggregated data reverses or differs after conditioning on a grouping variable.
- How it is used
- Frames causal decisions by showing why simple comparisons of treatments, prices, or strategies can mislead with observational data.
- How deep it goes
- Explains reversals through stratified examples, confounding, causal graphs, adjustment, and regression, without developing the adjustment formula fully.
- Examples
- Uses kidney stone recovery rates and synthetic hotel pricing data to expose reversals before and after conditioning on confounders.
- What you can do after
- Diagnose misleading associations by stratifying data, identifying confounders, and adding appropriate variables to regression models.
Read how Causal Inference for Data Science explains Simpson's paradox →
Statistics Slam Dunk
Simpson's paradox is a statistical phenomenon in which an association observed in aggregated data reverses or conflicts with the associations found in relevant subpopulations.
- How it is used
- Reveals how aggregation can conceal or contradict relationships within meaningful subpopulations.
- How deep it goes
- Introduces the phenomenon briefly and demonstrates visual detection, without developing a formal proof or causal explanation.
- Examples
- Uses NBA team payrolls, regular-season wins, playoff factors, and the 2012–13 season.
- What you can do after
- Diagnose/debug with it: compare pooled and subgroup correlations before interpreting an apparent trend.
Related concepts
- correlationcontrasts with · Correlation is a statistical relationship describing how two or more variables or quantities vary together, including the strength and direction of their association, without by itself implying causation.
- graphused by · A graph is a mathematical structure consisting of vertices connected by edges that represent relationships or connections.
- causal adjustmentcontrasts with · Causal adjustment estimates a causal effect by conditioning on variables that block noncausal pathways and averaging over their target-population distribution.
- subpopulationcontrasts with · A defined subset of a broader population that shares specified characteristics or conditions.
- confounderaddresses · A confounder is a variable associated with both an explanatory or treatment variable and an outcome, whose influence can distort their apparent relationship or estimated causal effect.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.