Expected improvement

As explained in Bayesian Optimization in Action

Expected improvement is a Bayesian optimization acquisition function that quantifies the expected amount by which evaluating a candidate may improve on the best observed objective value.

Expected improvement (EI) is a Bayesian optimization acquisition function for expensive objectives. For each candidate, it scores the expected amount by which an evaluation could beat the incumbent—the best objective value observed so far. The next query is the candidate with the largest score. section 4.0 section 4.2

EI turns a simple optimization preference into a probabilistic decision: favor points that may improve on the current best result, but value a large possible gain more than a tiny one. Under a Gaussian-process belief about unseen objective values, this expected improvement can be computed in closed form. section 4.0 section 4.3

Figure 4.15 The difference between PoI (left) and EI (right). The former only cares whether we improve from the incumbent, while the latter considers how much improvement is made.

Compare EI with probability of improvement (PoI): PoI asks only whether a candidate can beat the incumbent, whereas EI also weights how far it could surpass it. Thus a candidate with a larger potential gain can outrank one with only a marginal improvement. section 4.3

This emphasis on magnitude helps EI avoid PoI’s tendency toward overly exploitative, risk-averse choices. EI naturally uses both predicted value and uncertainty: promising high-value regions support exploitation, while uncertain regions can remain attractive because they may yield a substantial improvement. section 1.2.2 section 4.5

Hyperparameter tuning

Suppose the best validation accuracy seen for a neural network is 90%. EI ranks candidate hyperparameter settings by their expected improvement over that benchmark, then evaluates the highest-ranked setting. In an exercise with a limited query budget, EI is compared with PoI by tracking when each first reaches at least 90% accuracy. section 4.0 section 4.4.2

EI belongs in an iterative loop: fit a GP to evaluated points, maximize the EI score, evaluate the objective at that recommendation, add the observation, and repeat. Early in a search, many uncertain locations can score well; after a region has been examined sufficiently, other regions may become more valuable. section 1.2.2 section 1.2.4

Sequential EI selects one next point. Simply taking several highest-scoring points for a parallel batch is wasteful because they can cluster near the same maximum. Batch EI instead evaluates improvement through the best outcome within a batch under a multivariate Gaussian belief. section 7.1.2 section 7.2.1

BoTorch setup

For a maximization problem, construct analytic EI with a fitted model and the current maximum observed target value as best_f; then optimize the acquisition score to choose the next evaluation. section 4.3 section 4.5

How Bayesian Optimization in Action covers it

How deep it goes

Develops EI from improvement intuition through probabilistic magnitude, closed-form scoring, BoTorch use, and a batch extension.

Examples

Uses validation-accuracy tuning, a one-dimensional objective search, and BoTorch acquisition construction.

What it assumes

Assumes a Gaussian-process predictive belief while motivating improvement scoring from the incumbent.

What you can do after

Apply it via BoTorch to select and optimize the next Bayesian optimization query.

How it connects

Connects Gaussian processes, acquisition functions, Probability of Improvement, exploration–exploitation, and batch optimization.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  AI-Powered SearchExpected improvement is a Bayesian-optimization acquisition function that scores candidates by their expected gain over the current best result while accounting for predictive uncertainty.

Compare how 2 books explain expected improvement →

Related concepts

• constrained expected improvementincludes · Constrained expected improvement is a Bayesian optimization acquisition function whose score is the expected improvement multiplied by the probability that the candidate satisfies the constraints.
• ExpectedHypervolumeImprovementincludes · ExpectedHypervolumeImprovement is a BoTorch acquisition-function class that computes analytic expected hypervolume improvement for multiobjective Bayesian optimization.
• Bayesian optimizationkind of · Bayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.
• policykind of · A policy is a rule, strategy, or mapping that governs or determines decisions, actions, permissions, or behavior under specified conditions.
• acquisition functionkind of · An acquisition function is a scoring rule that ranks candidate evaluations by their expected usefulness for selecting the next sample or experiment.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.