Random search

As explained in Bayesian Optimization in Action

An optimization strategy that evaluates randomly selected candidate locations without adaptively modeling the objective.

Random search optimizes a black-box objective by selecting candidate points randomly within its domain and evaluating them. Uniform sampling gives every location equal probability. section 1.2.4

The basic procedure is:

• Sample candidate locations from the objective domain, usually uniformly. section 1.2.4
• Evaluate the objective at each sampled location until the evaluation budget is exhausted. section 1.2.4
• Return the sampled point with the best observed objective value. section 1.2.4

One-dimensional objective

For an objective defined from −5 to 5 with ten permitted evaluations, uniformly sampled points may cluster near an unpromising region. One trial found its best point near x = 4, with f(x) = 3.38, while potentially wasting evaluations around 0. section 1.2.4

Figure 1.6 The objective function that is to be maximized, where random search wastes resources on unpromising regions

Visualizes how random sampling can spend valuable evaluations in regions that produce poor objective values. section 1.2.4

How Bayesian Optimization in Action covers it

How deep it goes

Defines uniform and non-uniform sampling and illustrates resource waste, but stops short of implementation details.

Examples

Uses a one-dimensional function on [−5, 5], ten evaluations, and a best observed point near x = 4.

What you can do after

Apply random search by sampling candidate locations, evaluating them, and retaining the best observed value.

How it connects

Contrasts random search with grid search and Bayesian optimization.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Designing Deep Learning Systems: A guide for software engineersA hyperparameter search method that samples configurations randomly from a defined search space until a search budget is exhausted.

Compare how 2 books explain random search →

Related concepts

• grid searchalternative to · Grid search is a search method that evaluates configurations formed from specified candidate values, typically every combination in their Cartesian product.
• Hyperbandalternative to · Hyperband is a hyperparameter-optimization algorithm that adaptively allocates computational resources among configurations while successively eliminating poorly performing candidates.
• lower confidence bounduses · A lower confidence bound is an acquisition function that combines a predicted response with a multiple of predictive uncertainty to favor promising or informative evaluations.
• training datasetuses · A collection of examples or observations supplied to a learning procedure to fit or adapt a model or learn its patterns or parameters.
• Bayesian optimizationcontrasts with · Bayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.
• Hyperoptimplemented by · Hyperopt is an open-source Python library for automated hyperparameter optimization over configurable parameter spaces and search algorithms.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.