11 The effect of a time-related event
This chapter covers
- Synthetic controls, regression discontinuity designs and differences in differences
- Understanding the assumptions and limitations of these three methods
- Using these methods to estimate causal effects
In chapter 1, we used an example of evaluating the effect of a newly designed website. We learned that the best way to make such an evaluation was using A/B tests. However, as we already know, it is not always possible to perform A/B tests. Therefore, in this final chapter, we revisit how to assess the decision effect by comparing the situation before and after without relying on A/B tests. That is, we focus on understanding the effect of time-related events.
In this chapter, we’ll explore three techniques for estimating the effect of a decision. Each technique works for a different situation. We’ll stick to the following example as our guide; as we explore each technique, we’ll modify the example accordingly to understand when the technique works, what assumptions it has, and how to use it to estimate the ATE.