liveBook · Manning

Evaluation period

As explained in Causal Inference for Data Science

An evaluation period is the selected post-event interval over which an intervention's effect is assessed.

An evaluation period is the post-event interval chosen to measure an intervention’s effect. Start immediately after the event when effects are immediate; move the start later when effects are delayed. section 11.3.6

Choosing a campaign window

For a T-shirt social-media campaign, delivery time makes immediate measurement misleading, so the evaluation period should begin several days after launch. The chosen window should also reflect whether the goal is a short-term, medium-term, or long-term effect. section 11.3.6

Table 11.3 Consequences of choosing endpoints farther from time event T. The Direction arrow shows how the timepoints move with respect to T.
P b start
P b end
T
P a start
P a end
← Direction
Training data window
• More sample size
• Potentially less accuracy due to context change
← Direction
Event anticipation
• Smaller sample size
• Potentially more accuracy with less context change
→ Direction
Delayed effects
• Less data to evaluate the impact
• Potentially less accuracy due to context change
→ Direction
Long-term impact evaluation
• More data to evaluate the impact
• Potentially less accuracy due to context change

Compare how moving the endpoints changes sample size, contextual accuracy, and the data available for delayed or long-term effects. section 11.3.6

How Causal Inference for Data Science covers it

How deep it goes
Explains immediate, delayed, novelty, and sustained effects, but stops short of a general optimization rule for window selection.
Examples
Uses a delayed-delivery marketing campaign and contrasts whole-period evaluation with evaluation at the event time.
What it assumes
Assumes an identified event time and an estimated counterfactual outcome.
What you can do after
Apply it via CausalImpact by defining pre-event and post-event periods that match the intervention’s expected timing.
How it connects
Connects evaluation periods with causal effects, counterfactuals, event anticipation, delayed effects, and training data.
Cover of Causal Inference for Data Science
liveBookmanning.com
Read Causal Inference for Data Science on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Causal Inference for Data Science. The section links point to the parts of the book it draws on.