Sobol sequence

As explained in Bayesian Optimization in Action

A Sobol sequence is a low-discrepancy numerical sequence that distributes points evenly across a multidimensional domain.

A Sobol sequence is an infinite list of points designed to cover a multidimensional Euclidean region evenly. It provides a practical candidate set when dense grids become intractable and ordinary uniform sampling leaves coverage gaps. section 5.3.2

Use Sobol points to support approximate optimization over a search space:

• Generate candidates in the unit cube with a Sobol engine. section 5.3.2
• Scale the candidates to the targeted search bounds. section 5.5
• Evaluate a Gaussian-process sample on the candidates and select the point with the highest sampled value for Thompson sampling. section 5.3.2
• Request multiple selected points for batch optimization by setting the sample count to the batch size. section 7.3

One-dimensional candidate generation

In PyTorch, create 1,000 scrambled Sobol candidates with `torch.quasirandom.SobolEngine(1, scramble=True)` and `sobol.draw(1000)`. For a search interval from −5 to 5, transform the unit-cube output with `10 * candidate_x − 5` before applying Thompson sampling. section 5.3.2

Figure 5.14 Points from a Sobol sequence vs. uniformly sampled points in the two-dimensional unit square. The Sobol sequence covers the square more evenly and, therefore, should be used by TS.

Compare Sobol and uniformly sampled points in a two-dimensional unit square; the more even Sobol coverage motivates its use for candidate generation. section 5.3.2

How Bayesian Optimization in Action covers it

How deep it goes

Defines the sequence, compares its coverage with uniform sampling, and demonstrates generation and scaling without deriving its construction.

Examples

Uses unit-square coverage plots, a one-dimensional PyTorch candidate set, hyperparameter tuning, and a four-dimensional airplane-design search.

What it assumes

Assumes familiarity with search spaces, Gaussian-process models, Thompson sampling, and objective-function evaluation.

What you can do after

Apply it via PyTorch and BoTorch to generate scaled candidates for sequential or batch optimization.

How it connects

Connects Sobol sequences to Thompson sampling, Gaussian processes, Bayesian optimization, and batch optimization.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsA quasi-random low-discrepancy sequence designed to distribute points evenly throughout a multidimensional sample space.

Compare how 2 books explain Sobol sequence →

Related concepts

• samplingalternative to · Sampling is a procedure for selecting or generating observations, values, or outcomes from a larger population, signal, or probability distribution according to a specified method.
• SobolEngineimplemented by · SobolEngine is a PyTorch quasi-random sampling class that generates Sobol-sequence points in a unit cube.
• max-value entropy searchused by · Max-value Entropy Search is a Bayesian optimization acquisition policy that selects evaluations expected to reduce uncertainty about the global maximum value of an objective.
• Thompson samplingused by · Thompson sampling is a Bayesian sequential decision method that samples from posterior beliefs about uncertain rewards or outcomes and selects an action according to the sampled values.
• search spaceuses · A search space is the set of possible states, choices, configurations, or candidate solutions that may be considered when exploring or solving a problem.
• sequencekind of · A sequence is an ordered collection of elements whose positions determine their order of traversal or processing.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.