liveBook · Manning

Sampling

As explained in Bayesian Optimization in Action

Uniform sampling draws points so that equal-size regions of the sampling domain have equal probability.

Uniform sampling draws points so that equal-size regions of a domain have equal probability. In optimization, random search samples candidate evaluations uniformly within the objective domain, without favoring any region in advance. section 1.2.4

Sampling supports several practical activities:

  • Explore an objective function when evaluations are limited, while recognizing that random points can cluster and waste evaluations. section 1.2.4
  • Compare random search with grid search: grids spread evaluations evenly, while random sampling may discover useful regions without a fixed layout. section 1.2.4
  • Draw function realizations from a Gaussian-process predictive distribution to visualize possible behaviors beyond its mean and confidence interval. section 2.4.4

Sampling an optimization domain

Suppose an objective is evaluated at most 10 times over [-5, 5]. Random search samples locations uniformly and evaluates the objective there; one run finds a point near x = 4 with value 3.38. A different run could cluster elsewhere, so uniform sampling may spend resources in an unpromising region. section 1.2.4

torch.manual_seed(0)
    samples = predictive_distribution.sample(torch.Size([5]))

Reproduce five GP function samples with a fixed random seed using the predictive distribution's sample method. section 2.4.4

How Bayesian Optimization in Action covers it

How deep it goes
Defines uniform sampling, illustrates clustering and wasted evaluations, and demonstrates GP sampling in Python, stopping short of sampling theory.
Examples
Uses bounded objective search, clustered random evaluations, and five plotted samples from a zero-mean Gaussian process.
What it assumes
Assumes an objective domain or predictive distribution, while introducing uniform probability and GP sampling through concrete examples.
What you can do after
Apply it via random search for bounded objectives and use GPyTorch predictive distributions to generate reproducible function samples.
How it connects
Connects random search and grid search with Bayesian optimization, and connects function sampling with Gaussian processes and predictive distributions.
Cover of Bayesian Optimization in Action
liveBookmanning.com
Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

Compare how 23 books explain sampling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.