Linear cost model

As explained in Bayesian Optimization in Action

A linear cost model represents evaluation cost as a fixed component plus a weighted term that varies linearly with fidelity.

A linear cost model estimates the known cost of querying a fidelity as a fixed cost plus a weighted correlation value. Higher-fidelity approximations usually resemble the ground truth more closely and therefore cost more. section 9.3.1 section 9.7

For fidelity correlation c, fixed cost b, and weight w, the cost is b + wc. The model predicts no unknown quantity: it encodes costs already specified by the user so optimization can trade information against expense. section 9.3.1 section 9.7

Multiple low-fidelity approximations

With fixed cost 0.2 and weight 1, fidelities 0.1, 0.3, and 1 cost 0.3, 0.5, and 1.2 respectively. In a Branin optimization problem, this makes cheaper approximations available before querying the true objective. section 9.6

Figure 9.9 The linear cost model for multifidelity optimization. The cost of querying a data point on a fidelity scales linearly with the correlation between that fidelity and the ground truth f(x).

Use the figure to visualize the positive linear relationship between fidelity correlation and query cost. section 9.3.1

How Bayesian Optimization in Action covers it

How deep it goes

Defines the fixed-plus-weighted formula and demonstrates numeric costs, while deferring nonlinear cost-model implementation.

Examples

Uses BoTorch configuration and a Branin problem with two low-fidelity approximations at correlations 0.1 and 0.3.

What it assumes

Assumes fidelity correlations are stored as feature columns and distinguishes their data quality from their query cost.

What you can do after

Apply it via BoTorch's AffineFidelityCostModel to assign query costs during multifidelity acquisition optimization.

How it connects

Connects fidelity correlation, Gaussian processes, multifidelity optimization, maximum-value entropy search, and inverse-cost weighting.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Related concepts

• fidelityconfigures · Fidelity is the degree to which an output, model, or representation accurately reflects or is supported by its relevant source, target, evidence, or behavior.
• linear regressionkind of · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.