Gradient-based one-side sampling
Gradient-based one-side sampling is a gradient-boosting technique that retains examples with large gradients while subsampling and reweighting examples with small gradients.
Ensemble Methods for Machine Learning
Gradient-based one-sided sampling (GOSS) is a gradient-boosting sampling technique that retains examples with large gradients while subsampling and reweighting examples with small gradients.
- How it is used
- Speeds LightGBM training on large datasets by reducing the number of examples used for tree learning.
- How deep it goes
- Explains gradient-based selection, representative sampling, and reweighting, but stops short of implementation details or formal derivation.
- Examples
- Uses document retrieval and compares GOSS with random forests, standard gradient-boosted decision trees, and DART.
- What you can do after
- Apply GOSS through LightGBM when large training sets make example processing a bottleneck, while checking generalization.
Read how Ensemble Methods for Machine Learning explains gradient-based one-side sampling →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
Gradient-based one-side sampling is a sampling algorithm that retains examples with the largest gradients and randomly samples examples with smaller gradients using corrective weights.
- How it is used
- Reduces gradient-boosting training time by limiting the examples processed at each stage.
- How deep it goes
- Explains gradient-prioritized sampling and its accuracy trade-off without detailing the weighting procedure or implementation.
- Examples
- Contrasts retaining large-gradient examples with randomly sampling smaller-gradient examples.
- What you can do after
- Apply it through LightGBM to reduce processed training examples while monitoring predictive-performance effects.
Related concepts
- LightGBMused by · LightGBM is an open-source gradient-boosting framework for efficiently training tree-based machine-learning models.
- gradient boostingalternative to · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.
- samplingkind of · Sampling is a procedure for selecting or generating observations, values, or outcomes from a larger population, signal, or probability distribution according to a specified method.
- downsamplingkind of · Downsampling is the reduction of data quantity, spatial resolution, temporal resolution, or sampling rate by retaining, selecting, or aggregating fewer representative values or observations.
- diversityproduces · Diversity is the degree of variation among members of a set or population in relevant characteristics, perspectives, behaviors, or outcomes.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.