Hyperparameter search space

Q: How deeply does Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI cover hyperparameter search space?

Explains candidate values, boundaries, conditional parameters, and grid-size trade-offs but stops short of full search-algorithm theory.

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A hyperparameter search space is the set of candidate values and constraints from which model configurations are selected during optimization.

A hyperparameter search space defines which settings an optimization procedure may test and the allowed values or boundaries for each setting. It turns parameter tuning from ad hoc experimentation into a controlled search tied to evaluation metrics, cross-validation, and a search strategy. section 6.3 section 6.3.1

A useful search space specifies:

The hyperparameters whose effects matter for the problem, together with discrete candidates, continuous ranges, or categorical alternatives. section 6.3 section 6.3.1
Constraints or conditional adjustments when selected values require different parameters, as with XGBoost boosters and objectives. section 7.2.5
A size appropriate to the available computation: exhaustive grids suit few discrete parameters, while broader spaces can use adaptive search. section 6.3.1 section 7.2.5

XGBoost regression space

An Optuna study can test booster and objective categories alongside ranges for estimator count, learning rate, subsampling, column sampling, and tree depth. It evaluates each suggested configuration by cross-validated mean absolute error over 60 trials, then reports the best settings. section 7.2.5

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

liveBook manning.com

Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Designing Deep Learning Systems: A guide for software engineersThe set of allowable values or value ranges from which hyperparameters may be selected during optimization.

Compare how 2 books explain hyperparameter search space →

Related concepts

hyperparameter tuningused by · Hyperparameter tuning is the process of selecting or adjusting configurable model or procedure settings and evaluating them against a chosen objective.
hyperparameter gridpart of · A hyperparameter grid is a structured set of candidate hyperparameter values whose combinations define configurations for systematic evaluation.
Optunaused by · Optuna is an open-source Python framework for automated hyperparameter optimization through repeated trials that explore parameter configurations.
grid searchused by · Grid search is a search method that evaluates configurations formed from specified candidate values, typically every combination in their Cartesian product.
search spacekind of · A search space is the set of possible states, choices, configurations, or candidate solutions that may be considered when exploring or solving a problem.
hyperparameterhas part · A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.