11 The effect of a time-related event

 

This chapter covers

  • Synthetic controls, regression discontinuity designs and differences in differences
  • Understanding the assumptions and limitations of these three methods
  • Using these methods to estimate causal effects

In chapter 1, we used an example of evaluating the effect of a newly designed website. We learned that the best way to make such an evaluation was using A/B tests. However, as we already know, it is not always possible to perform A/B tests. Therefore, in this final chapter, we revisit how to assess the decision effect by comparing the situation before and after without relying on A/B tests. That is, we focus on understanding the effect of time-related events.

In this chapter, we’ll explore three techniques for estimating the effect of a decision. Each technique works for a different situation. We’ll stick to the following example as our guide; as we explore each technique, we’ll modify the example accordingly to understand when the technique works, what assumptions it has, and how to use it to estimate the ATE.

11.1 Which types of data will we use?

11.2 Regression discontinuity design

11.2.1 Data simulation

11.2.2 RDD terminology

11.2.3 Assumptions

11.2.4 Effect estimation

11.2.5 RDD in practice

11.3 Synthetic controls

11.3.1 Data simulation

11.3.2 Synthetic controls terminology

11.3.3 Assumptions

11.3.4 Effect estimation

11.3.5 Synthetic controls in practice

11.3.6 Selecting training and predicting time periods

11.4 Differences in differences

11.4.1 Data simulation

11.4.2 DiD terminology

11.4.3 Assumptions

11.4.4 Effect estimation

11.4.5 In practice

11.5 Chapter quiz

11.6 Method comparison

11.7 References