A control metric is a health check for an experiment. Unlike a key metric, which the change aims to improve, it should remain stable. A meaningful shift can signal flawed assignment, degraded user experience, model failure, or an external change that makes the result unreliable. section 12.2.3 section 12.3.1
Use control metrics to monitor the experiment’s operating conditions:
- Check traffic balance and assignment integrity, such as views per variation or the number of visitors. section 12.2.3 section 12.4.2
- Watch service health, including response latency, queries per second, and customer support reports. section 12.4.2
- Track business or model operations that should not unexpectedly move, such as checks per day, update frequency, and offline metrics. section 12.4.1
- Investigate segment-level fairness and engagement changes before trusting the primary result. section 12.3.1
Search experiment
A search-engine A/B test may target click-through rate and conversion rate, while controlling views per variation, response latency, query volume, search engagement, and support reports. A latency increase or support spike can expose a broken experience even if conversion has not yet fallen. section 12.4.2
Control metrics do not decide whether the experiment succeeds; they establish whether its comparison remains credible. If they deteriorate, investigate or stop early, then record the failure and add safeguards for future tests. section 12.2.3 section 12.3.5
