Robustness is the ability of a model to maintain acceptable behavior when deployment conditions differ from training conditions. A model becomes fragile when it relies on correlations that change across environments rather than relationships that remain stable. section 1.3.3
Three failure modes make robustness a practical modeling concern:
- Overfitting gives excessive weight to spurious statistical patterns; causal modeling redirects learning toward patterns grounded in causal relationships. section 1.3.3
- Underspecification leaves multiple models equally successful on test data but differently effective after deployment; identification determines whether assumptions and data imply a unique causal answer. section 1.3.3
- Data drift changes deployment data over time; causal invariance can preserve performance when the underlying relationship remains stable. section 1.3.3
Robustness also concerns causal estimates. Backdoor adjustment depends on blocking every backdoor path, so an omitted confounder can introduce bias, reverse the apparent treatment effect, or create an effect where none exists. Sensitivity analysis tests whether modest unobserved-confounder influence changes the estimate substantially. section 11.5.5
Stress-testing an estimated treatment effect
A DoWhy workflow adds an unobserved common cause with varying influence on treatment and outcome, then examines the resulting effect estimates in a heatmap. A stable estimate changes little; a sensitive estimate changes sharply as the assumed confounder influence increases. section 11.5.5
Figure 11.11 A heatmap illustrating the effects of adding an unobserved confounder on the ATE estimate
Use the heatmap to inspect how assumed unobserved-confounder influence changes the average treatment effect estimate. section 11.5.5
