Exploitation is the optimization choice to focus new evaluations on regions that current evidence predicts will perform well. In Bayesian optimization, a policy uses observations and a Gaussian-process model to select the next expensive black-box evaluation, while balancing this focus against exploration of uncertain regions. section 4.0 section 4.1.1
Use exploitation as one side of a deliberate decision trade-off:
- Favor inputs with high observed or predicted objective values so optimization makes progress near known good regions. section 4.1.2
- Avoid pure exploitation when limited observations could hide a better region elsewhere in the search space. section 4.1.2
- Tune the balance in Upper Confidence Bound (UCB): smaller β weights the predictive mean more heavily and makes the policy more exploitative; larger β encourages exploration through uncertainty. section 5.2.2
- Compare improvement policies: Probability of Improvement can be overly exploitative and risk-averse, while Expected Improvement typically balances exploration and exploitation more effectively. section 4.5
Known favorite versus an uncertain alternative
At a restaurant, ordering a reliably good burger is exploitation; trying an unfamiliar fish or steak is exploration. Likewise, after two Forrester-function observations near x = 1 and x = 2 produce values around 1.6 and 1.5, a policy can favor that promising area—but should account for what remains unknown. section 4.1.2
The same trade-off appears in multi-armed bandits. Pulling the slot-machine arm with the best observed success rate exploits current reward evidence, but repeatedly doing so can miss an arm whose true reward rate is higher but still uncertain. section 5.1.1
Figure 5.8 Different upper bounds of a GP, corresponding to different CIs and β values. The larger β is, the more explorative the UCB policy becomes.
Compare UCB upper bounds as β changes: the selected maximum shifts because the policy gives uncertainty more or less influence relative to predictive mean. section 5.2.2
