Probability of improvement

As explained in Bayesian Optimization in Action

Probability of Improvement is a Bayesian optimization acquisition function that assigns each candidate the probability that its objective value will exceed a specified incumbent or improvement threshold.

Probability of Improvement (PoI) is a Bayesian optimization acquisition policy for expensive black-box objectives. At each iteration, it scores each candidate by the chance that its predicted objective value will beat the current incumbent, then queries the candidate with the highest score. section 4.0 section 4.2.2

For a maximization problem, improvement means exceeding the best observed value. A Gaussian process supplies a normal predictive distribution at each candidate; PoI is the area of that distribution above the incumbent threshold. Equivalently, compute one minus the normal CDF at the incumbent, using the predictive mean and standard deviation. section 4.2.2

Figure 4.11 Flowchart of how the PoI score is computed. By following this procedure, we can compute how likely any candidate point is to be able to improve from the incumbent.

Follow the score from a candidate’s GP prediction through its normal-distribution tail area to its probability of improving on the incumbent. section 4.2.2

Use PoI in the optimization loop as follows:

• Fit a Gaussian process to the observations collected so far and identify the best observed objective value. section 4.5
• Construct PoI with that incumbent as its improvement threshold, score candidate inputs, and maximize the acquisition score. section 4.2.2 section 4.5
• Evaluate the selected input, add its label to the data, and repeat until the available function-evaluation budget is exhausted. section 4.2.3

PoI favors any positive improvement, however small. That makes it risk-averse and prone to exploitation: it can repeatedly sample a promising local region rather than inspect uncertain regions that may contain a better optimum. Raising the required improvement by ε makes the policy more exploratory, but choosing an appropriate threshold can be difficult across objectives. section 4.2.3 section 4.4.1

Escaping a local optimum

On the one-dimensional Forrester function, unmodified PoI remains between 0 and 2 and converges near a local optimum, missing the global optimum around 4. Setting a minimum improvement threshold, such as ε = 0.1, changes which candidates qualify as improvements and can improve exploration. section 4.2.3 section 4.4.1

In BoTorch, create an analytic ProbabilityOfImprovement policy with a model and best_f equal to the incumbent, then pass it to optimize_acqf() to locate the next query. For a batch, the improvement event is defined by whether the best objective value among batch members exceeds the incumbent. section 4.2.2 section 7.2.1

How Bayesian Optimization in Action covers it

How deep it goes

Derives PoI as a normal-distribution tail area, implements it in BoTorch, and examines its iterative behavior and epsilon modification.

Examples

Uses validation accuracy, alloy stability, the one-dimensional Forrester function, and two-dimensional SVM hyperparameter tuning.

What you can do after

Apply it via BoTorch to optimize a PoI acquisition score and adjust epsilon when exploitation stalls progress.

How it connects

Connects Gaussian processes, acquisition functions, exploration–exploitation, Expected Improvement, and batch Bayesian optimization.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Related concepts

• batch optimizationused by · Batch optimization is an optimization setting in which multiple candidate evaluations are selected or executed together.
• ProbabilityOfImprovementimplemented by · ProbabilityOfImprovement is a BoTorch acquisition-module class that computes probability-of-improvement scores from a Gaussian process and an incumbent value.
• cumulative distribution functionuses · A cumulative distribution function gives the probability that a random variable is less than or equal to each specified value.
• incumbentuses · An incumbent is the currently established or best-known solution, product, or approach against which alternatives are evaluated.
• policykind of · A policy is a rule, strategy, or mapping that governs or determines decisions, actions, permissions, or behavior under specified conditions.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.