liveBook · Manning

Exploit-only greedy strategy

As explained in Optimization Algorithms: AI techniques for design, planning, and control problems

An exploit-only greedy strategy repeatedly selects the action with the highest estimated mean reward after initial action trials.

An exploit-only greedy strategy tries each slot machine once, estimates its mean reward, then repeatedly selects the machine with the highest estimate. It can become stuck after an unlucky early result because it stops exploring. section 12.1.6

Three-machine example

If the first payoffs from machines 1, 2, and 3 are 7, 6, and 3, the strategy favors machine 1 thereafter, even though machine 2 may have the highest true mean reward. section 12.1.6

A practical implementation follows these steps:

  • Try every machine once, estimate each machine's mean reward, and select the machine with the largest estimate on later trials. section 12.1.6
  • Compare the result with epsilon-greedy and UCB strategies when premature commitment is costly. section 12.1.6

How Optimization Algorithms: AI techniques for design, planning, and control problems covers it

How deep it goes
Explains initial trials, greedy selection, regret, and implementation, but stops short of uncertainty-aware decision rules.
Examples
Uses three slot machines, first-trial payoffs, payoff distributions, NumPy listings, and average-regret outputs.
What it assumes
Assumes familiarity with actions, rewards, estimated means, and regret while treating the bandit as a single-state decision problem.
What you can do after
Apply via NumPy by sampling payoffs, selecting the highest estimated mean, and measuring average regret over trials.
How it connects
Connects multi-armed bandits with exploration, exploitation, regret, epsilon-greedy, and upper confidence bound strategies.
Cover of Optimization Algorithms: AI techniques for design, planning, and control problems
liveBookmanning.com
Read Optimization Algorithms: AI techniques for design, planning, and control problems on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Optimization Algorithms: AI techniques for design, planning, and control problems. The section links point to the parts of the book it draws on.