Randomization assigns experiment units to control or treatment variants by chance, creating a controlled comparison for measuring a model’s product impact. Choose the unit deliberately—user, account, session, or device—and verify that observed traffic matches the planned allocation. section 6.3.5
Assignment is not exposure. A treatment-assigned user may never reach the model surface, may receive a fallback, or may see cached output. Log both assignment and actual exposure before interpreting outcome metrics. section 6.3.5
Validate a randomized experiment before trusting its results:
- Confirm that users were assigned randomly using the intended unit and that the traffic split matches the planned allocation. section 6.3.5
- Record variant assignment, model version, feature set, ranking logic, post-processing, exposure, fallbacks, and cached responses. section 6.3.5
- Check overlapping experiments, metric consistency, pre-planned segments, and decision rules before examining results. section 6.3.5
A lightweight model experiment
Extend an existing feature-flag system to randomize exposure to model versions, associate assignments and results with a user or session ID, and measure key metrics over time. Hosted services can provide randomization, assignment, exposure logging, and metrics pipelines when building infrastructure in-house is impractical. section 6.7.2






