Upper confidence bound
Upper confidence bound is a decision rule that combines an estimated value with an uncertainty bonus to select actions or candidates optimistically.
Bayesian Optimization in Action
An upper confidence bound is a Bayesian optimization acquisition policy that combines a predictive mean with an uncertainty-based confidence term to select candidate evaluations.
- How it is used
- Selects the next Bayesian optimization query by optimistically scoring predicted objective values under uncertainty.
- How deep it goes
- Derives μ + βσ from Gaussian-process confidence intervals, explains β behavior, and implements UCB with BoTorch.
- Examples
- Uses weather uncertainty, slot machines, the one-dimensional Forrester function, and support-vector-machine hyperparameter tuning.
- What you can do after
- Apply it via BoTorch by configuring UpperConfidenceBound with a GP model and β, then optimizing its acquisition score.
Read how Bayesian Optimization in Action explains upper confidence bound →
Optimization Algorithms: AI techniques for design, planning, and control problems
Upper confidence bound is an action-selection strategy that combines estimated reward with an uncertainty bonus to balance exploitation and exploration.
- How it is used
- Balances exploration and exploitation when selecting actions in a noncontextual multi-armed bandit.
- How deep it goes
- Implements initialization, a confidence-score formula, iterative updates, parameter tuning, and regret comparison through Python code.
- Examples
- Uses three slot machines, 90 trials, payoff distributions, NumPy listings, and average-regret comparisons.
- What you can do after
- Apply it via NumPy by selecting arms, updating rewards, tuning c, and measuring average regret.
Related concepts
- acquisition functionuses · An acquisition function is a scoring rule that ranks candidate evaluations by their expected usefulness for selecting the next sample or experiment.
- UCB parameterconfigured by · A UCB parameter is a positive scalar that weights predictive uncertainty in an upper confidence bound score and controls its exploration tendency.
- policykind of · A policy is a rule, strategy, or mapping that governs or determines decisions, actions, permissions, or behavior under specified conditions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.