Optimal value

As explained in Bayesian Optimization in Action

An optimal value is the objective-function value attained at an optimizer.

In Bayesian optimization, the optimal value is f* = f(x*): the objective-function value attained at the optimizer. Under a Gaussian-process belief, f* is itself a random variable. section 6.2.1

Although the optimizer x* can be a complicated, non-Gaussian vector, f* is always a one-dimensional real value. Sampling its distribution therefore makes entropy-based search easier to approximate. section 6.2.1

Incumbent bound

If the best observed objective value is 1.6, the distribution of f* is truncated around 1.6 because the optimum cannot be worse than the incumbent. section 6.2.1

Figure 6.11 The GP belief (top left), the distribution of the function optimizer x* (bottom), and the distribution of the optimal value f* (right). The distribution of the optimal value is always one-dimensional and, therefore, easier to work with than that of the optimizer.

Compare the complicated distribution of the optimizer with the simpler, always one-dimensional distribution of the optimal value. section 6.2.1

How Bayesian Optimization in Action covers it

How deep it goes

Defines f* formally, explains its GP-induced distribution, and connects samples to expected posterior-entropy reduction.

Examples

Uses an incumbent value of 1.6 and visual comparisons between optimizer and optimal-value distributions.

What it assumes

Assumes Gaussian-process beliefs, objective functions, and optimizers while clarifying how scalar optimal values relate to optimizer vectors.

What you can do after

Apply it via entropy search by sampling the optimal-value distribution and selecting queries that maximize expected entropy reduction.

How it connects

Connects optimal value with optimizers, Gaussian processes, Bayesian optimization, entropy search, posterior entropy, and acquisition functions.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Related concepts

• optimizerproduced by · An optimizer is an input location at which an objective function attains its maximum or minimum over the domain under consideration.
• optimizationproduced by · Optimization is the systematic selection or adjustment of values, actions, or system designs to improve a specified objective, often subject to constraints.
• valuekind of · A value is the content, quantity, information, or worth attributed to an entity, property, expression, or outcome.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.