Refutation tests whether a causal conclusion survives attempts to violate its assumptions. Unlike predictive models, causal models make claims about how the data-generating process works, so evidence that contradicts those claims should trigger revision rather than confidence from fit alone. section 4.3 section 11.5
Apply refutation at several levels of a causal workflow:
- Test a causal DAG by checking whether observed dependence violates conditional independences implied by d-separation. section 4.3
- Compare predicted intervention outcomes with outcomes from feasible real-world interventions; mismatches refute the model. section 7.6.2
- Stress-test an estimated effect by simulating violations of the causal assumptions behind identification and the statistical assumptions behind the estimator. section 11.4.6 section 11.5
In DoWhy, CausalModel.refute_estimate runs refuters as simulation-based statistical tests. Each refuter attacks a different assumption; a p-value below .05 is treated as evidence that the assumptions are refuted. A failed test does not merely lower confidence—it identifies pressure to revisit the model, data, or estimator. section 11.5
Prioritizing real-world tests
Enumerate feasible interventions, predict their outcomes, rank the most surprising predictions, and run those interventions first. Contradictions produce earlier, more informative model updates while conserving resources. section 7.6.2
Figure 10.4 The causal inference workflow. The identification step is an essential step in the workflow.
Use the causal inference workflow to place refutation after identification and estimation, before accepting the result. section 10.2.2
