Covariate shift occurs when the contexts reaching a decision system change over time: traffic may move from desktop to mobile, or new customer segments may appear. The model now faces inputs unlike those seen during training, even when the underlying outcome relationship may remain stable. section 5.6
In contextual bandits, respond to covariate shift by:
- Monitoring feature distributions so the policy can detect when current traffic no longer resembles historical data. section 5.6
- Keeping an exploration floor—a small, permanent amount of exploration—to reduce blind spots after the context distribution changes. section 5.6 section 5.7
- Treating the system as continuous experimentation rather than a one-time A/B test, because business environments keep changing. section 5.6 section 5.7
Traffic mix changes
A policy trained mostly on desktop visitors may encounter predominantly mobile visitors later. Monitoring the changed feature mix and preserving exploration helps it continue learning instead of committing blindly to old decisions. section 5.6

