A search space is the set of candidate configurations an optimization process can explore. In hyperparameter optimization, each dimension represents a tunable input such as layer width, learning rate, batch size, or epochs. section 5.5.2 section 5.4.2
Adding dimensions expands the number of candidates multiplicatively: four parameters with ten possible steps each create 10,000 grid-search cells. Evolutionary strategies can explore these dimensions while fitness records reveal clustering around promising configurations. section 5.5.2
Implementations make the space explicit by:
- Mapping normalized individual or particle values onto parameter ranges, while static generators hold selected parameters fixed. section 5.4.2
- Discretizing continuous candidates and caching evaluations to reduce duplicate model training. section 9.5
Four-dimensional hyperparameter space
Varying middle-layer size, learning rate, batch size, and epochs creates a four-dimensional space. Setting ten candidate steps for each parameter yields 10,000 possible grid-search configurations. section 5.5.2
