liveBook · Manning

Function evaluation

As explained in Bayesian Optimization in Action

A function evaluation, or query, is the computation of an objective function's output at a specified input.

A function evaluation, also called a query, computes an objective function’s output for a specified input. In black-box optimization, each trial tests a particular setting and observes its result, often at substantial time or computational cost. section 1.1.2

Function evaluations determine how an optimization loop uses available resources:

  • Sequential optimization makes one query, receives its function evaluation, and then selects the next query. section 7.0
  • Batch optimization makes multiple queries in parallel, increasing experimental throughput when processing units or physical resources can be parallelized. section 7.0
  • Selecting several nearby high-scoring points can waste evaluations by clustering experiments around essentially one input location. section 7.1.2

Airplane design optimization

A simulated airplane objective is evaluated at points in a four-dimensional unit hypercube. An experiment can use 100 total function evaluations, arranged as batches of five, and record the best value found as optimization proceeds. section 7.5

Figure 1.2 The framework of a black box optimization problem. We repeatedly query the function values at various locations to find the global optimum.

The diagram shows the repeated-query framework: evaluate objective-function values at different locations while searching for the global optimum. section 1.1.2

How Bayesian Optimization in Action covers it

How deep it goes
Defines evaluation as objective output at an input and contrasts sequential and parallel use, without deriving evaluation internals.
Examples
Uses hyperparameter combinations, parallel experiments, and a simulated four-dimensional airplane-design objective.
What it assumes
Assumes an objective function and candidate settings, while defining the query operation from the beginning.
What you can do after
Apply it via Bayesian optimization to count, sequence, batch, and assess objective-function experiments.
How it connects
Connects objective functions, black-box optimization, Bayesian optimization, hyperparameter tuning, acquisition functions, and batch optimization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.