Suppose an online store adds a feature and sales rise. Did the feature cause the increase, or did both change for another reason? Causal inference turns questions like this into specific queries and uses data to answer them. Unlike ordinary statistical inference, it constrains the analysis with explicit causal assumptions: correlation alone cannot establish what an intervention would change. section 1.3.1 section 1.1
A practical workflow separates the causal question from the statistical estimate:
- Pose a query. Name the treatment and outcome, and specify the intervention to compare. An average treatment effect compares outcomes under two treatment levels; a conditional average treatment effect makes that comparison for a specified group or condition. section 11.1 section 1.5.3
- Build a causal model. A directed acyclic graph (DAG) records assumptions about what causes what. Start with variables central to the query, then consider common causes and variables useful for the analysis, such as confounders, instruments, mediators, and effect modifiers. section 3.3.1
- Check identification before estimating. Ask whether the query has a unique answer given the model’s assumptions and available data. If not, obtain additional measurements, reconsider the assumptions, or leave the question unanswered. section 10.2.2
- Estimate the identified query using an appropriate statistical method, then test its sensitivity to violations of the assumptions that supported the result. Available estimation approaches include regression, propensity scores, Bayesian inference, and double machine learning. section 10.2.2
Engagement is not experiment assignment
A game randomly assigns some players to modified mechanics intended to encourage side quests. Comparing the assigned groups estimates the effect of changing the mechanics on purchases. It does not directly answer how much high rather than low side-quest engagement changes purchases: engagement can also depend on other, potentially confounding factors. The treatment named in the query determines which effect must be identified and estimated. section 11.1
Effects are only one kind of causal query. Causal models can also support intervention prediction, counterfactual reasoning, explanation, and attribution. When a proposed experiment cannot be run, observational data may help estimate its effect or guide which experiment to run. Stronger models can answer additional questions, but their extra assumptions create more ways to be wrong: a structural causal model’s claim about how one variable changes another is harder to validate than a DAG’s claim that it causes another. section 1.6 section 6.1.5
Figure 10.4 The causal inference workflow. The identification step is an essential step in the workflow.
The workflow diagram places identification between modeling the causal assumptions and estimating the query; it also includes a final refutation step. section 10.2.2
