liveBook · Manning

Hyperparameter tuning

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Hyperparameter optimization, or tuning, is the process of searching for hyperparameter settings that improve a model's evaluation objective.

Hyperparameter tuning searches for settings that improve a model’s evaluation objective. A reliable process defines the search space, uses cross-validation, and selects a search algorithm that balances quality, computation, and time. section 6.3

A practical tuning workflow includes:

  • Define the evaluation metric and the hyperparameters whose effects matter. section 6.3
  • Build cross-validation so promising settings generalize beyond the available data rather than overfitting it. section 6.3
  • Choose a search strategy that finds suitable settings within practical computational cost. section 6.3

Tuning an XGBoost regressor

For the Tokyo Airbnb regression problem, Optuna evaluates 60 trials using cross-validated mean absolute error. It explores boosters, objectives, estimators, learning rate, sampling ratios, and tree depth. One resulting configuration uses a gbtree booster, 901 estimators, depth seven, and a Tweedie objective, reducing MAE from the initial attempt. section 7.2.5

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

How deep it goes
Explains search-space design, cross-validation, algorithm choice, and a 60-trial optimization, without deriving search algorithms mathematically.
Examples
Uses a Tokyo Airbnb XGBoost regressor, Optuna trials, MAE, booster choices, objectives, and parameter-importance visualizations.
What it assumes
Assumes an evaluation metric and model are available, while establishing search-space and cross-validation choices explicitly.
What you can do after
Apply it via Optuna to evaluate candidate XGBoost configurations and retrieve suitable hyperparameters for a regression objective.
How it connects
Connects evaluation metrics, cross-validation, Bayesian optimization, Optuna, XGBoost, boosting objectives, and regularization.
Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
liveBookmanning.com
Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Also explained in

Compare how 18 books explain hyperparameter tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.