Surrogate model

Q: How deeply does Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI cover surrogate model?

Explains updates, acquisition, exploration, exploitation, and model choices, but does not derive their mathematics.

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A surrogate function is an approximate model of an objective function that predicts objective behavior across untested parameter settings.

A surrogate model is an approximate model of an objective function. In Bayesian optimization, it predicts model performance across untested hyperparameter settings, reducing the need to evaluate every combination directly. section 6.3.4

A practical Bayesian optimization loop works as follows:

Build or update the surrogate from feedback produced by the objective function. section 6.3.4
Use an acquisition function to balance exploration of uncertain settings with exploitation of promising settings. section 6.3.4
Choose the next hyperparameters, evaluate the model, and repeat the fine-tuning cycle. section 6.3.4

Tuning an XGBoost classifier

An Optuna study can search settings such as learning rate, tree depth, subsampling, and regularization, then report the best parameters and validation score after 60 trials.

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

liveBook manning.com

Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Experimentation for EngineersA surrogate function is a model that approximates a measured response as a function of parameter values and estimates the response at unmeasured values.
liveBook manning.com

Bayesian Optimization in ActionA surrogate model is a predictive model that approximates an objective function from observed evaluations.
liveBook manning.com

Outlier Detection in PythonA proxy model is an approximate model that imitates the behavior of another model or detector for analysis or explanation.
liveBook manning.com

Ensemble Methods for Machine LearningA surrogate model is an interpretable approximation trained to reproduce the behavior of a more complex model in a specified region or representation.

Compare how 5 books explain surrogate model →

Related concepts

Optunaused by · Optuna is an open-source Python framework for automated hyperparameter optimization through repeated trials that explore parameter configurations.
Bayesian optimizationused by · Bayesian optimization is a sequential method for optimizing an expensive or unknown objective by using a probabilistic model to guide evaluations.
interpretabilityused by · Interpretability is the degree to which people can understand a model’s behavior, outputs, predictions, or the factors that produce them.
black boxuses · A black box is a system or component whose internal workings are treated as opaque while its inputs, outputs, or observable behavior are considered.
Gaussian processproduced by · A Gaussian process is a probability distribution over functions in which every finite collection of function values has a joint Gaussian distribution.
labelcontrasts with · A label is a value or annotation associated with an item that identifies its target category, outcome, or reference answer for learning or evaluation.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.