Uncertainty is the lack of certainty about an unknown quantity or function. In hyperparameter tuning, observed results do not determine how performance behaves at untested values, so uncertainty must guide the next experiment. section 1.1.1
Bayesian optimization represents uncertainty across a search space and uses it alongside predicted performance. A policy therefore balances exploration of regions with uncertain predictions against exploitation of regions with high predicted means. section 4.1
Uncertainty supports several practical decisions:
- Estimate whether an untested hyperparameter value could outperform observed settings. section 1.1.1
- Explore search-space regions where predictions are uncertain, rather than repeatedly sampling only the current best-looking region. section 4.1
- Choose searches that reduce uncertainty about an unknown quantity; entropy measures how much uncertainty remains after evidence is gathered. section 6.1.2
Finding a lost remote
If a remote is believed to be uniformly located somewhere in the living room, searching part of that room narrows the possible locations. Finding it removes the uncertainty; searching the bathroom leaves the belief unchanged when the remote is absent there. section 6.1.2
Figure 4.1 The exploration–exploitation tradeoff in BayesOpt. Each policy needs to decide whether to query regions with high uncertainty (exploration) or to query regions with a high predicted mean (exploitation).
A visual comparison of exploration in high-uncertainty regions and exploitation near a high predicted mean makes the decision tradeoff concrete. section 4.1
