Multi-parameter optimization

As explained in Bayesian Optimization in Action

Multiobjective optimization simultaneously optimizes multiple objectives, generally seeking solutions that represent trade-offs among them.

Multi-parameter, or multiobjective, optimization seeks inputs that score well on several objectives at once. It matters when optimizing one goal alone harms another: a predictive model may be accurate yet too large or costly for real-time use. section 11.0 section 11.1

Conflicting objectives usually rule out one universally best point. A point is nondominated when improving one objective requires worsening at least one other. The nondominated points form the Pareto frontier, the boundary of attainable trade-offs; no point lies beyond it in every objective. section 11.2 section 11.5

Figure 11.6 The Pareto frontier that traces through the nondominated points. No data point lies beyond (above and to the right of) this Pareto frontier.

Compare objective values directly: dominated points fall behind another point, while stars mark the trade-offs retained on the Pareto frontier. section 11.2

For expensive black-box objectives, Bayesian optimization keeps one Gaussian process (GP) model per objective. The models support joint reasoning about candidate inputs while leaving most of the usual BayesOpt workflow intact. section 11.1 section 11.5

Dominated hypervolume measures the objective-space region between a reference point and the Pareto frontier. Larger hypervolume indicates a stronger set of trade-offs. Expected Hypervolume Improvement (EHVI) uses GP predictions to choose evaluations expected to expand that region. section 11.2 section 11.5

Two conflicting one-dimensional objectives

A Forrester-based objective peaks near x = 4.5, while a second objective peaks near x = −4.5. Joint optimization therefore collects points between these extremes rather than selecting only either single-objective optimum. section 11.2

Airplane aerostructural design

Optimize flight utility and negated flight cost over a four-dimensional unit search space. Fit GP models, use a reference point of [−1.5, −2], and compare EHVI with alternating expected improvement through average hypervolume. section 11.4

How Bayesian Optimization in Action covers it

How deep it goes

Defines nondominance, Pareto frontiers, reference-point hypervolume, multiple GP models, and EHVI, then applies them in a multidimensional exercise.

Examples

Uses conflicting one-dimensional functions, hypervolume comparisons, and four-dimensional airplane aerostructural design.

What it assumes

Assumes familiarity with the BayesOpt workflow and GP modeling while motivating multiobjective trade-offs from concrete problems.

What you can do after

Apply it via BoTorch to compute hypervolume and use EHVI for competing expensive objectives.

How it connects

Connects Bayesian optimization, Gaussian processes, Pareto frontiers, hypervolume, expected improvement, and exploration versus exploitation.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Related concepts

• objective functionuses · An objective function is a function or criterion that assigns values to candidate solutions or parameter settings and is optimized by minimizing or maximizing those values.
• ExpectedHypervolumeImprovementused by · ExpectedHypervolumeImprovement is a BoTorch acquisition-function class that computes analytic expected hypervolume improvement for multiobjective Bayesian optimization.
• Gaussian processuses · A Gaussian process is a probability distribution over functions in which every finite collection of function values has a joint Gaussian distribution.
• tradeoffcontrasts with · A tradeoff is a situation in which improving one objective or property requires accepting a cost, limitation, or reduction in another.
• pairwise comparisonused by · A pairwise comparison evaluates two alternatives against one another to determine their relative preference, value, or rank.
• Bayesian optimizationuses · Bayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.