liveBook · Manning

Epsilon-greedy decay algorithm

As explained in Experimentation for Engineers

A variant of the epsilon-greedy algorithm whose exploration probability decreases according to the number of measurements.

The epsilon-greedy decay algorithm balances exploration and exploitation while reducing exploration as measurements accumulate. Its exploration probability follows a fixed schedule based on n, not observed performance. section 3.1.3

Operationally, the method:

  • Stops when epsilon falls below 0.01, then accepts B when its mean CTR exceeds A's; otherwise it retains A. section 3.1.3

Candidate-change evaluation

A run can stop after fewer measurements than a comparable A/B test. Repeated evaluations reported higher mean CTR for epsilon_greedy_decay() than for ab_test(), while its prescribed stopping time distinguished it from ordinary epsilon-greedy.

How Experimentation for Engineers covers it

How deep it goes
Explains the decay schedule, stopping rule, regret behavior, and tuning requirement, but does not detail finite-time safeguards.
Examples
Uses competing arms A and B, CTR measurements, a 0.01 stopping threshold, and comparisons against A/B testing.
What it assumes
Assumes observed arm means and CTR comparisons, while supplying the exploration schedule and stopping procedure.
What you can do after
Operate/configure the algorithm, tune c, stop at a prescribed exploration threshold, and select or reject candidate changes.
How it connects
Connects epsilon-greedy decay with multi-armed bandits, A/B testing, CTR, exploration, exploitation, and regret.
Cover of Experimentation for Engineers
liveBookmanning.com
Read Experimentation for Engineers on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Experimentation for Engineers. The section links point to the parts of the book it draws on.