Exposure logging records whether an assigned user or unit actually received the changed model output. Assignment alone is insufficient: users may trigger a surface while the model fails, falls back, or serves cached results. Without exposure data, analysis can treat untreated users as treated and weaken the launch decision. section 6.3.5
Validate exposure before interpreting experiment metrics:
- Record the assigned variant and whether the user actually saw the model output. section 6.3.5
- Identify the served model version, feature set, ranking logic, prompt, or post-processing rule. section 6.3.5
- Capture fallbacks to control models, cached results, default rules, or empty responses. section 6.3.5
- Check overlapping experiments and consistent success and guardrail metric instrumentation. section 6.3.5
A lightweight exposure record
Extend a feature flag system to randomize model versions and log assignment, exposure, and results under a user or session ID. This creates a basic controlled experiment without requiring a full experimentation platform. Hosted services such as Optimizely, Eppo, Statsig, and Split.io can provide randomization, assignment, exposure logging, and metrics pipelines, although integration with model-serving and data systems still requires engineering work. section 6.7.2 section 6.3.5
