Smoothness

As explained in Bayesian Optimization in Action

Smoothness is the property that nearby or similar inputs produce correlated and therefore similar function values.

Smoothness means that similar inputs are correlated, so nearby points tend to produce similar function values. In a Gaussian process (GP), conditioning on an observation updates predictions around that observation rather than at one isolated point. section 2.3

The covariance function, also called the kernel, sets this relationship and therefore controls the GP’s smoothness. A large length scale produces smoother predictions and less uncertainty; different dimensions can use different length scales. section 3.4 section 3.6

A nearby observation

After observing a high function value at 3, a smooth GP raises its predictions at 2.9 and 3.1 because those inputs are close to 3. The same behavior appears when conditioning on several observations: the posterior mean smoothly interpolates the data while uncertainty decreases around observed points. section 2.3

Use covariance choices to express different assumptions about smoothness:

• Increase the length scale to model slower output variation and smoother predictions. section 3.6
• Use automatic relevance determination when different input dimensions vary at different rates. section 3.6
• Use the Matérn kernel’s parameter nu to represent different levels of prediction smoothness. section 3.6

Figure 2.14 Predictions made by a GP conditioned on zero, one, two, and four observations

Compare GP predictions after conditioning on zero, one, two, and four observations to see smooth interpolation and local uncertainty reduction. section 2.3

How Bayesian Optimization in Action covers it

How deep it goes

Explains nearby-point correlation, posterior behavior, covariance control, length scales, ARD, and Matérn smoothness without deriving kernels.

Examples

Uses observations at 3 with nearby points 2.9 and 3.1, plus conditioned GP prediction plots and a Forrester-function dataset.

What it assumes

Assumes familiarity with Gaussian processes and covariance functions while building smoothness intuition from nearby observations.

What you can do after

Apply covariance functions and length-scale choices to control GP smoothness and interpret local uncertainty.

How it connects

Connects Gaussian processes with covariance functions, kernels, length scales, automatic relevance determination, and Matérn kernels.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Geometry for ProgrammersSmoothness is the degree to which a mathematical function or geometric object lacks discontinuities, sharp joins, or abrupt changes in direction.

Compare how 2 books explain smoothness →

Related concepts

• covariance functionconfigured by · A covariance function, also called a kernel, specifies the covariance between function values at pairs of inputs in a Gaussian process.
• length scalecontrasts with · A length scale is a positive parameter that determines the distance over which a function’s correlations or similarities substantially change.
• order of continuitymeasured by · The order of continuity is the highest derivative order through which a function and its derivatives remain continuous.
• derivativeuses · A derivative is a mathematical quantity that measures the instantaneous rate of change of a function with respect to an input variable.
• Gaussian processused by · A Gaussian process is a probability distribution over functions in which every finite collection of function values has a joint Gaussian distribution.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.