liveBook · Manning

Covariate shift

As explained in Applied Reinforcement Learning

Covariate shift is a distribution change in which the distribution of input contexts changes while the conditional outcome relationship may remain stable.

Covariate shift occurs when the contexts reaching a decision system change over time: traffic may move from desktop to mobile, or new customer segments may appear. The model now faces inputs unlike those seen during training, even when the underlying outcome relationship may remain stable. section 5.6

In contextual bandits, respond to covariate shift by:

  • Monitoring feature distributions so the policy can detect when current traffic no longer resembles historical data. section 5.6
  • Keeping an exploration floor—a small, permanent amount of exploration—to reduce blind spots after the context distribution changes. section 5.6 section 5.7
  • Treating the system as continuous experimentation rather than a one-time A/B test, because business environments keep changing. section 5.6 section 5.7

Traffic mix changes

A policy trained mostly on desktop visitors may encounter predominantly mobile visitors later. Monitoring the changed feature mix and preserving exploration helps it continue learning instead of committing blindly to old decisions. section 5.6

How Applied Reinforcement Learning covers it

How deep it goes
Defines context drift and gives monitoring and exploration guidance, but does not derive weighting methods or model internals.
Examples
Uses desktop-to-mobile traffic changes and the appearance of new customer segments as concrete context shifts.
What you can do after
Diagnose shifting inputs by monitoring feature distributions and maintain an exploration floor to reduce policy blind spots.
How it connects
Connects covariate shift with contextual bandits, exploration, non-stationarity, personalization, and continuous experimentation.

Also explained in

Compare how 2 books explain covariate shift →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.