Epsilon-greedy
Epsilon-greedy is an action-selection strategy that chooses a random action with a specified probability and otherwise chooses the action with the highest estimated value.
Experimentation for Engineers
Epsilon-greedy is a bandit allocation method that usually selects the currently best alternative while exploring alternatives at a specified probability.
- How it is used
- Optimizes business metrics while evaluating production system variants instead of allocating traffic equally.
- How deep it goes
- Defines the decision rule, supplies Python simulations, covers decayed stopping, and generalizes from two alternatives to multiple arms.
- Examples
- Uses ad click models with CTR rewards and a social-media recommender that randomly shows some posts.
- What you can do after
- Apply it via a Python/NumPy-style policy to allocate among alternatives while tracking observed rewards and tuning exploration.
Read how Experimentation for Engineers explains epsilon-greedy →
Applied Reinforcement Learning
An epsilon-greedy strategy selects a random action with probability epsilon and otherwise selects an action with the highest estimated value.
- How it is used
- Balances exploration and exploitation in repeated bandit decisions and supplies a non-contextual baseline for personalized discounting.
- How deep it goes
- Works through action selection and incremental reward updates in an agent implementation, then compares regret and contextual alternatives.
- Examples
- Uses simulated ad click-through rates and a discount-revenue baseline to demonstrate action selection.
- What you can do after
- Builds an epsilon-greedy bandit agent from scratch using random action selection and incremental reward estimates.
Read how Applied Reinforcement Learning explains epsilon-greedy →
Optimization Algorithms: AI techniques for design, planning, and control problems
An action-selection strategy that chooses a random action with a specified probability and otherwise selects the currently best-known action.
- How it is used
- Balances exploration and exploitation when selecting actions in multi-armed bandit problems.
- How deep it goes
- Explains the selection rule, loop steps, epsilon parameter, regret calculation, and comparison with alternative bandit strategies.
- Examples
- Uses three slot machines, 90 trials, payoff distributions, NumPy code, and average-regret comparisons.
- What you can do after
- Apply it via NumPy by implementing random exploration, greedy exploitation, payoff sampling, and regret tracking.
Build a Robo-Advisor with Python (From Scratch)
Epsilon-greedy is an action-selection strategy that chooses a random action with probability epsilon and a currently best-known action otherwise.
- How it is used
- Balances exploration and exploitation while Q-learning learns financial-planning actions across simulated episodes.
- How deep it goes
- Explains probabilities, trade-offs, tie-breaking, Q updates, and episode generation, while deferring more sophisticated exploration schedules.
- Examples
- Uses wealth-state transitions, action-value arrays, random tie-breaking, and one million generated episodes.
- What you can do after
- Apply it via Python to generate exploratory Q-learning paths and update action values.
Read how Build a Robo-Advisor with Python (From Scratch) explains epsilon-greedy →
What to know first
- multi-armed banditA multi-armed bandit is a sequential decision problem in which an agent repeatedly chooses among alternatives with uncertain rewards, balancing reward seeking with information gathering.
- click-through rateClick-through rate is the proportion of impressions or opportunities to display an item that result in a click on that item.
- dynamic programmingDynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
Related concepts
- multi-armed banditaddresses · A multi-armed bandit is a sequential decision problem in which an agent repeatedly chooses among alternatives with uncertain rewards, balancing reward seeking with information gathering.
- SARSA(λ)used by · SARSA(λ) is an on-policy SARSA variant that uses eligibility traces to propagate temporal-difference credit to previously visited state-action pairs.
- Q-learningused by · Q-learning is a model-free reinforcement-learning algorithm that estimates state-action values through updates based on rewards and subsequent state values.
- Thompson samplingcontrasts with · Thompson sampling is a Bayesian sequential decision method that samples from posterior beliefs about uncertain rewards or outcomes and selects an action according to the sampled values.
- counterfactual outcomeproduces · A counterfactual outcome is a hypothetical result that would have occurred under a condition, decision, or intervention different from the one actually observed.
Questions about epsilon-greedy
What is epsilon-greedy?
Epsilon-greedy is an action-selection strategy that chooses a random action with a specified probability and otherwise chooses the action with the highest estimated value.
Which Manning books explain epsilon-greedy?
Experimentation for Engineers, Applied Reinforcement Learning, Optimization Algorithms: AI techniques for design, planning, and control problems and Build a Robo-Advisor with Python (From Scratch).
What should I know before learning epsilon-greedy?
Manning books that teach epsilon-greedy usually introduce multi-armed bandit, click-through rate, dynamic programming and mean first.
What is the difference between epsilon-greedy and Thompson sampling?
Epsilon-greedy is an action-selection strategy that chooses a random action with a specified probability and otherwise chooses the action with the highest estimated value. Thompson sampling is a Bayesian sequential decision method that samples from posterior beliefs about uncertain rewards or outcomes and selects an action according to the sampled values.
What are the parts of epsilon-greedy?
Epsilon-greedy includes exploration rate.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



