Max-value entropy search

As explained in Bayesian Optimization in Action

Max-value Entropy Search is a Bayesian optimization acquisition policy that selects evaluations expected to reduce uncertainty about the global maximum value of an objective.

Max-value Entropy Search (MES) is a Bayesian optimization acquisition policy for expensive black-box objectives. Rather than directly model the optimizer location x*, it asks which evaluation will most reduce uncertainty about the global maximum value f*. This target is always one-dimensional, even when the input space has many dimensions, making it more tractable than reasoning directly about the optimizer’s distribution. section 6.2.1 section 6.4

Figure 6.11 The GP belief (top left), the distribution of the function optimizer x* (bottom), and the distribution of the optimal value f* (right). The distribution of the optimal value is always one-dimensional and, therefore, easier to work with than that of the optimizer.

The optimizer distribution can be complicated and non-Gaussian, while the distribution of f* remains one-dimensional; MES uses the latter as a practical proxy. section 6.2.1

MES scores a candidate by its approximate expected reduction in posterior entropy over f*. A strong score can combine a high predicted mean with high uncertainty, so the policy balances exploitation of promising regions with exploration of uncertain ones. Sampling from the distribution of f* approximates the score because a closed-form calculation is infeasible. section 6.2.1 section 6.4

A practical MES loop is:

• Fit or update a Gaussian process model of the objective from observed evaluations. section 6.2.2
• Generate candidate points, commonly with a Sobol sequence that covers the search space. section 6.2.2
• Initialize qMaxValueEntropy with the model and candidate set; more candidates improve approximation fidelity at greater computational cost. section 6.2.2
• Optimize the acquisition score, evaluate the selected point, and repeat until the evaluation budget is exhausted. section 6.2.2

num_candidates = 1000                                    #1
sobol = torch.quasirandom.SobolEngine(1, scramble=True)  #1
candidate_x = sobol.draw(num_candidates)                 #1
 
candidate_x = 10 * candidate_x - 5                       #2
 
policy = botorch.acquisition.max_value_entropy_search
.qMaxValueEntropy(                                     #3
    model, candidate_x                                   #3
)                                                        #3
 
with torch.no_grad():                                    #4
    acquisition_score = policy(xs.unsqueeze(1))          #4

The implementation constructs Sobol candidates over the search range, initializes qMaxValueEntropy, and evaluates acquisition scores with a GP model. section 6.2.2

Hyperparameter tuning and batch selection

For an SVM accuracy surface over penalty C and RBF-kernel parameter γ, MES can drive a two-dimensional Bayesian optimization loop using a larger Sobol candidate set. In batch optimization, initialize MES similarly but use cyclic acquisition optimization to choose a batch that maximizes posterior information about f*; the GP model must support fantasizing about future observations. section 6.3.2 section 7.4.2

MES also has a multifidelity variant that combines information about the optimal value with query cost in a return-on-investment measure. This lets the policy trade knowledge against cost across low- and high-fidelity evaluations; a reported Forrester comparison shows multifidelity MES outperforming its single-fidelity counterpart. section 9.4

How Bayesian Optimization in Action covers it

How deep it goes

Defines entropy-reduction scoring, explains sampling-based approximation, and implements sequential and batch policies without deriving closed-form scores.

Examples

Uses a one-dimensional Forrester function, SVM hyperparameter tuning, airplane design optimization, and multifidelity Forrester optimization.

What it assumes

Assumes a Gaussian-process Bayesian optimization pipeline and introduces MES through its optimal-value information objective.

What you can do after

Apply it via BoTorch’s qMaxValueEntropy with a GPyTorch model and Sobol candidate set in sequential or batch optimization.

How it connects

Connects Bayesian optimization with Gaussian processes, entropy search, Sobol sequences, batch optimization, and multifidelity optimization.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Related concepts

• multifidelity max-value entropy searchcontrasts with · Multifidelity Max-value Entropy Search is a Bayesian optimization policy that selects both input locations and fidelity levels to reduce uncertainty about the objective's maximum while accounting for evaluation cost.
• Sobol sequenceuses · A Sobol sequence is a low-discrepancy quasi-random sequence designed to distribute points evenly across a multidimensional domain.
• Bayesian optimizationkind of · Bayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.
• cyclic optimizationimplemented by · Cyclic optimization is an iterative optimization method that successively refines individual components or members of a candidate solution until convergence.
• information theoryuses · Information theory is the mathematical study of quantifying information and uncertainty, and of representing, encoding, storing, processing, and communicating information.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.