A training dataset is the portion of observations used to fit a model. Set aside a separate test dataset, fit on the training data, and compare predictions with known outcomes to estimate performance on new data. section 4.1
Choose the training window to balance predictive relevance against the amount of data available:
- Use observations from a stable context so relationships remain predictive. section 11.3.6
- Move the window’s beginning farther from the event to gain sample size, accepting possible loss of accuracy from context change. section 11.3.6
- End the window before treatment or event effects appear; anticipation can make even nominally pretreatment observations unsuitable. section 11.3.6
- Try alternative endpoints and use cross-validation to select the window with the lowest predictive error. section 11.3.6
Movie-release counterfactual
To simulate viewership without a new movie’s release, train on an earlier pretreatment period. If viewers begin catching up on older movies before release, exclude that anticipated-effect period even though the official event has not occurred. section 11.3.6
Table 11.3 Consequences of choosing endpoints farther from time event T. The Direction arrow shows how the timepoints move with respect to T.
|
P
b
start
|
P
b
end
|
T
|
P
a
start
|
P
a
end
|
|---|---|---|---|---|
|
← Direction
Training data window
• More sample size
• Potentially less accuracy due to context change
|
← Direction
Event anticipation
• Smaller sample size
• Potentially more accuracy with less context change
|
→ Direction
Delayed effects
• Less data to evaluate the impact
• Potentially less accuracy due to context change
|
→ Direction
Long-term impact evaluation
• More data to evaluate the impact
• Potentially less accuracy due to context change
|
Compare how moving training and evaluation endpoints changes sample size, contextual accuracy, and the ability to assess delayed or long-term effects. section 11.3.6

















































