Random search
Random search is an optimization technique that explores a search space by evaluating candidate solutions or configurations selected through randomized choices.
Designing Deep Learning Systems: A guide for software engineers
A hyperparameter search method that samples configurations randomly from a defined search space until a search budget is exhausted.
- How it is used
- Deploys independent configuration trials to explore a hyperparameter space under a fixed optimization budget.
- How deep it goes
- Explains sampling, search spaces, budgets, and trial execution while deferring mathematical reasoning.
- Examples
- Uses learning-rate and batch-size ranges, a 100-trial budget, and a conditional classifier search.
- What you can do after
- Apply random search by defining an objective, specifying a search space, and running a bounded set of HPO trials.
Read how Designing Deep Learning Systems: A guide for software engineers explains random search →
Bayesian Optimization in Action
An optimization strategy that evaluates randomly selected candidate locations without adaptively modeling the objective.
- How it is used
- Uses random evaluation as a baseline for expensive black-box optimization before introducing adaptive methods.
- How deep it goes
- Defines uniform and non-uniform sampling and illustrates resource waste, but stops short of implementation details.
- Examples
- Uses a one-dimensional function on [−5, 5], ten evaluations, and a best observed point near x = 4.
- What you can do after
- Apply random search by sampling candidate locations, evaluating them, and retaining the best observed value.
Read how Bayesian Optimization in Action explains random search →
Related concepts
- grid searchalternative to · Grid search is a search method that evaluates configurations formed from specified candidate values, typically every combination in their Cartesian product.
- Hyperbandalternative to · Hyperband is a hyperparameter-optimization algorithm that adaptively allocates computational resources among configurations while successively eliminating poorly performing candidates.
- lower confidence bounduses · A lower confidence bound is an acquisition function that combines a predicted response with a multiple of predictive uncertainty to favor promising or informative evaluations.
- training datasetuses · A collection of examples or observations supplied to a learning procedure to fit or adapt a model or learn its patterns or parameters.
- Bayesian optimizationcontrasts with · Bayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.
- Hyperoptimplemented by · Hyperopt is an open-source Python library for automated hyperparameter optimization over configurable parameter spaces and search algorithms.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.