An exploration floor keeps an agent trying less-established actions at a small, permanent minimum rate. In contextual bandits, it hedges against changing traffic mixes, new customer segments, and other forms of covariate shift by preventing the policy from stopping exploration entirely. section 5.6 section 5.7
Changing traffic mix
Suppose a recommendation or advertising system learned mostly from desktop users, then receives more mobile traffic and new customer segments. A permanent exploration dose keeps testing alternatives instead of relying blindly on an increasingly outdated policy. This makes exploration a practical hedge against drift and a component of continuous experimentation rather than a one-time test. section 5.6 section 5.7
