Grid search

As explained in Bayesian Optimization in Action

Grid search is an optimization technique that evaluates points arranged on a regular grid over the search space.

Grid search explores an objective function by dividing its search space into evenly spaced segments and evaluating each segment endpoint. Spreading evaluations across the range avoids the accidental clustering that can waste resources in random search. section 1.2.4

A basic grid-search procedure is:

• Divide the search space into evenly spaced segments. section 1.2.4
• Evaluate the objective at each segment endpoint. section 1.2.4
• Select the best evaluated point, while recognizing that the true global optimum may lie between grid points. section 1.2.4

A one-dimensional search

Across a range from −5 to 5, grid search evaluates evenly spaced locations and identifies the endpoint at 5 as the best tested point, with an objective value of roughly 4.86. It improves on random search but still misses the actual global optimum and remains inefficient at narrowing a promising region. section 1.2.4

Figure 1.7 Grid search is still inefficient at narrowing down a good region.

Use the figure to see how evenly spaced evaluations can remain inefficient at narrowing down a good region. section 1.2.4

How Bayesian Optimization in Action covers it

How deep it goes

Defines segment partitioning and endpoint evaluation, but stops short of adaptive narrowing or reliable global-optimum discovery.

Examples

Evaluates evenly spaced points from −5 to 5 and selects the point at 5 with an objective value near 4.86.

What it assumes

Assumes a defined search space and evaluable objective, without requiring prior knowledge of important regions.

What you can do after

Apply it by partitioning a search space evenly and evaluating each segment endpoint.

How it connects

Contrasts grid search with random search and Bayesian optimization.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA model-selection technique that evaluates an exhaustive set of specified hyperparameter combinations.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksGrid search is a hyperparameter-optimization technique that evaluates configurations formed from a specified Cartesian product of candidate values.
• 

  liveBookmanning.com
  Designing Deep Learning Systems: A guide for software engineersA hyperparameter search method that evaluates configurations formed from the Cartesian product of specified candidate values.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningGrid search is a model-selection method that evaluates combinations from a specified set of parameter values to identify a performant configuration.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIGrid search is an exhaustive hyperparameter optimization technique that evaluates every combination of specified hyperparameter values.

Compare how 6 books explain grid search →

Related concepts

• hyperparameteruses · A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.
• cross-validationuses · Cross-validation is a resampling technique that evaluates a model by repeatedly fitting it on some portions of available data and assessing it on held-out portions.
• random searchcontrasts with · Random search is an optimization technique that explores a search space by evaluating candidate solutions or configurations selected through randomized choices.
• Bayesian optimizationcontrasts with · Bayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.
• linear regressionused by · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.