A control group is a comparable set of customers or leads that does not receive the tested intervention, or receives a designated baseline such as a generic message. Comparing its outcomes with a treatment group helps separate causal uplift from seasonal trends, competitor promotions, and other external factors. section 7.7.3
Use control groups to protect the credibility of experiments:
- Assign eligible participants randomly to treatment or control groups; use stratified randomization when important subgroups may behave differently. section 3.2.1
- Keep longitudinal participants in their original groups, include originally assigned participants in analysis, and wait through the planned duration. section 3.2.1
- Export only treatment members to marketing destinations when possible; keeping control members unexposed reduces contamination risk. section 3.2.1
- Consider spillover from word-of-mouth and use geo-experiments or matched-market tests when individual-level separation is unreliable. section 3.2.1
Testing an added SMS sequence
A treatment group receives an automated SMS sequence alongside existing email, while the control group receives only email. Higher conversions or revenue reveal the incremental effect of SMS. section 7.7.3
