A unit of randomization is the entity independently assigned to an experiment variant, such as a user, session, or account. Choosing it determines how consistently people experience the treatment and how interpretable the measured effect will be. section 6.3.2
Define the unit alongside the other mechanics that connect a hypothesis to rollout:
- Specify which entity receives assignment: user, session, account, or another operational unit. section 6.3.2
- Keep the experience consistent when consistency is needed for interpretation; user-level assignment gives one user the same recommendation experience during the test. section 6.3.6
- Interpret assignment with exposure rules, exclusion criteria, success metrics, and guardrail metrics rather than treating the unit choice as sufficient evidence of impact. section 6.3.2 section 6.3.6
Movie recommendation test
A recommendation experiment uses user-level randomization. Active users who watched at least one title in the past 14 days enter the test, while users already enrolled in another recommendation experiment on the same surface are excluded. Each user consistently receives the control model, Treatment A with conservatively weighted contextual features, or Treatment B with stronger weighting on recent searches and session context. section 6.3.6
