liveBook · Manning

Gradient-based one-side sampling

Gradient-based one-side sampling is a gradient-boosting technique that retains examples with large gradients while subsampling and reweighting examples with small gradients.

How 2 Manning books explain it

Ensemble Methods for Machine Learning

Gradient-based one-sided sampling (GOSS) is a gradient-boosting sampling technique that retains examples with large gradients while subsampling and reweighting examples with small gradients.

How it is used
Speeds LightGBM training on large datasets by reducing the number of examples used for tree learning.
How deep it goes
Explains gradient-based selection, representative sampling, and reweighting, but stops short of implementation details or formal derivation.
Examples
Uses document retrieval and compares GOSS with random forests, standard gradient-boosted decision trees, and DART.
What you can do after
Apply GOSS through LightGBM when large training sets make example processing a bottleneck, while checking generalization.

Read how Ensemble Methods for Machine Learning explains gradient-based one-side sampling →

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Gradient-based one-side sampling is a sampling algorithm that retains examples with the largest gradients and randomly samples examples with smaller gradients using corrective weights.

How it is used
Reduces gradient-boosting training time by limiting the examples processed at each stage.
How deep it goes
Explains gradient-prioritized sampling and its accuracy trade-off without detailing the weighting procedure or implementation.
Examples
Contrasts retaining large-gradient examples with randomly sampling smaller-gradient examples.
What you can do after
Apply it through LightGBM to reduce processed training examples while monitoring predictive-performance effects.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains gradient-based one-side sampling →

What to know first

Related concepts

Questions about gradient-based one-side sampling

What is gradient-based one-side sampling?

Gradient-based one-side sampling is a gradient-boosting technique that retains examples with large gradients while subsampling and reweighting examples with small gradients.

Which Manning books explain gradient-based one-side sampling?

Ensemble Methods for Machine Learning and Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI.

What should I know before learning gradient-based one-side sampling?

Manning books that teach gradient-based one-side sampling usually introduce gradient boosting first.

What is the difference between gradient-based one-side sampling and gradient boosting?

Gradient-based one-side sampling is a gradient-boosting technique that retains examples with large gradients while subsampling and reweighting examples with small gradients. Gradient boosting is an ensemble-learning technique that sequentially adds predictive models fitted to reduce a loss by following its gradient.

What is gradient-based one-side sampling a kind of?

Gradient-based one-side sampling is a kind of downsampling. Downsampling is the reduction of data quantity, spatial resolution, temporal resolution, or sampling rate by retaining, selecting, or aggregating fewer representative values or observations.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.