Gradient-based one-side sampling

Q: How deeply does Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI cover gradient-based one-side sampling?

Explains gradient-prioritized sampling and its accuracy trade-off without detailing the weighting procedure or implementation.

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Gradient-based one-side sampling is a sampling algorithm that retains examples with the largest gradients and randomly samples examples with smaller gradients using corrective weights.

Gradient-based one-side sampling (GOSS) speeds gradient boosting by reducing the number of training examples processed. It preserves examples with the largest gradients, which carry stronger learning signals, while randomly sampling examples with smaller gradients and applying corrective weights. section 5.5.2

Use GOSS to balance training speed against predictive performance:

Reduce row-processing cost when datasets contain many training examples. section 5.5.2
Avoid excessive sampling, which can harm training and decrease predictive performance. section 5.5.2
Distinguish row sampling from column reduction: GOSS reduces examples, whereas EFB reduces feature dimensionality. section 5.5.2

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

liveBook manning.com

Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Ensemble Methods for Machine LearningGradient-based one-sided sampling (GOSS) is a gradient-boosting sampling technique that retains examples with large gradients while subsampling and reweighting examples with small gradients.

Compare how 2 books explain gradient-based one-side sampling →

Related concepts

LightGBMused by · LightGBM is an open-source gradient-boosting framework for efficiently training tree-based machine-learning models.
gradient boostingalternative to · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.
samplingkind of · Sampling is a procedure for selecting or generating observations, values, or outcomes from a larger population, signal, or probability distribution according to a specified method.
downsamplingkind of · Downsampling is the reduction of data quantity, spatial resolution, temporal resolution, or sampling rate by retaining, selecting, or aggregating fewer representative values or observations.
diversityproduces · Diversity is the degree of variation among members of a set or population in relevant characteristics, perspectives, behaviors, or outcomes.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.