When a team asks whether side-quest engagement increases in-game purchases, logged player behavior provides evidence without assigning anyone to engage more or less. This is observational data: measurements of naturally occurring behavior, contrasted with data produced by an intervention or randomized experiment. section 7.1.3
Observational data records statistical associations arising from dependencies in the data-generating process. A group comparison can estimate an effect size, but the difference alone does not establish that changing the observed behavior would cause the outcome; causal identification is needed to assess whether observational data supports that inference. section 7.1.3 section 10.0
Use observational data to:
- Estimate associations when experiments cannot be run, then use causal models to simulate interventions and estimate causal effects. section 1.6
- Place logged observations at level 1 of the causal hierarchy, below interventional data and counterfactual data. section 10.1.3
- Check whether confounders and available causal structure permit identification before assuming that more data will solve the problem. section 10.0
Online game engagement and purchases
A database query returns 1,000 players: 482 with high side-quest engagement and 518 with low engagement. Mean purchases are $111.61 and $73.10, respectively. The $38.51 difference is a descriptive group contrast; deciding whether increasing engagement causes higher purchases requires causal analysis rather than this comparison alone. section 7.1.3
Table 7.1 Example rows from observational data on Side-Quest Engagement and In-Game Purchases
|
User ID
|
Side-Quest Engagement
|
In-Game Purchases
|
|---|---|---|
|
71d44ad5
|
high
|
156.77
|
|
e6397485
|
low
|
34.89
|
|
87a5eaf7
|
high
|
172.86
|
|
c5d78ca4
|
low
|
215.74
|
|
d3b2a8ed
|
high
|
201.07
|
|
dc85d847
|
low
|
12.93
|
Inspect representative rows pairing naturally observed side-quest engagement with in-game purchases. section 7.1.3


