A minimum detectable effect (MDE) is the smallest true change an experiment is designed to detect with specified statistical power and significance. It turns a product question into a measurable planning target. section 6.3.6
Use the MDE to plan and interpret an online experiment:
- Choose primary metrics and establish their baseline rates. section 6.3.6
- Calculate the required sample size with a power analysis rather than a generic rule of thumb. section 6.3.6
- Compare observed gains with the planned effect while checking guardrails, latency, cost, and satisfaction. section 6.3.6
Movie recommendation A/B test
For a four-week user-level test of contextual recommendation features, calculate sample size from baseline metrics and the MDE for primary outcomes. Treat flat overall results with segment improvement as a follow-up hypothesis, not an automatic launch decision. section 6.3.6

