liveBook · Manning

Gradient boosting

Also called GBM

Gradient boosting is an ensemble-learning technique that sequentially adds predictive models fitted to reduce a loss by following its gradient.

How 8 Manning books explain it

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A decision-tree ensemble model built by sequentially adding trees to optimize a predictive loss using gradient-based optimization.

How it is used
Applies sequential tree ensembles as a leading baseline for tabular classification, regression, ranking, and click-prediction tasks.
How deep it goes
Explains sequential corrections, logits, probabilities, tree controls, tuning tradeoffs, and inspectable XGBoost and LightGBM structures.
Examples
Demonstrates Airbnb NYC price classification, predicted-probability histograms, regression behavior, Scikit-learn cross-validation, and XGBoost and LightGBM tree extraction.
What you can do after
Apply it via Scikit-learn, XGBoost, or LightGBM to train, tune, evaluate, inspect, and compare tabular predictive models.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains gradient boosting →

Ensemble Methods for Machine Learning

Gradient boosting is an ensemble-learning technique that sequentially adds models trained to correct the errors of prior models using gradient-based optimization.

How it is used
Builds sequential tree ensembles that correct residual errors for classification, regression, and ranking.
How deep it goes
Explains residual intuition, gradient-descent correspondence, stepwise training, from-scratch trees, and LightGBM tuning.
Examples
Uses noisy two-moons classification and AutoMPG regression with Huber-loss LightGBM.
What you can do after
Build it from scratch with weak regression trees, residual fitting, line search, and additive prediction updates.

Read how Ensemble Methods for Machine Learning explains gradient boosting →

Grokking Machine Learning

An ensemble model that sequentially combines decision trees, with each tree improving the ensemble's fit to the data.

How it is used
Combines sequential decision trees into a strong learner and serves as a candidate in a real-world model comparison.
How deep it goes
Works through residual fitting and learning-rate updates; leaves classification adjustments to supplementary resources.
Examples
Fits weekly app-engagement days from user age and demonstrates classifier training on Titanic data.
What you can do after
Apply it via Scikit-Learn’s gradient-boosting classifier, then compare model performance using accuracy or F1-score.

Read how Grokking Machine Learning explains gradient boosting →

Build AI Drug Discovery Pipelines

Gradient boosting is an ensemble learning technique that sequentially adds weak models trained to reduce the current ensemble's loss or errors.

How it is used
Applies sequential ensemble learning to QSAR classification, especially when molecular descriptor data are tabular.
How deep it goes
Explains weak learners, residual-based updates, and the gradient rationale, then demonstrates XGBClassifier without detailing hyperparameter tuning.
Examples
Uses RNA-binding versus protein-binding compounds, Mordred descriptors, XGBClassifier, AUPRC, and feature importance by gain, coverage, and frequency.
What you can do after
Apply it via XGBClassifier to classify molecular compounds with stratified cross-validation and precision-recall evaluation.

Read how Build AI Drug Discovery Pipelines explains gradient boosting →

Transfer Learning for Natural Language Processing

An ensemble method that iteratively adds decision-tree-based models to address weaknesses in earlier models.

How it is used
Provides a decision-tree-based classification baseline for email spam and movie review tasks.
How deep it goes
Explains iterative learning and demonstrates classifier training and evaluation without deriving the boosting algorithm.
Examples
Uses email spam and IMDB movie review classification alongside classifier reports and an email ROC curve.
What you can do after
Apply gradient boosting via scikit-learn and assess a classifier using cross-validation and test metrics.

Read how Transfer Learning for Natural Language Processing explains gradient boosting →

Fight Fraud with Machine Learning

Gradient boosted trees are an ensemble model that sequentially trains decision trees, with each tree correcting errors from earlier trees.

How it is used
Detects fraudulent online transactions as a stronger tree-based alternative to random forest.
How deep it goes
Explains sequential error correction, hyperparameter tuning, feature importance, tree plots, and score aggregation into probabilities.
Examples
Uses XGBoost listings, four-tree plots, feature-importance plots, and an amount-threshold trace for transaction classification.
What you can do after
Apply it via XGBoost to train, tune, compare, and interpret a fraud classifier.

Read how Fight Fraud with Machine Learning explains gradient boosting →

Machine Learning System Design

A gradient-boosting model is an ensemble model built by sequentially fitting weak predictive models to reduce the errors of the current ensemble.

How it is used
Uses gradient boosting to improve retail demand forecasts while reducing the gap between delivered and sold items.
How deep it goes
Explains convergence, tree-count selection, and feature complexity, but does not derive the boosting algorithm mathematically.
Examples
Evaluates six quantile models for retail sales using convergence curves, tree-count grids, lag counts, and window sizes.
What you can do after
Evaluate gradient-boosting experiments with convergence checks, baseline comparisons, and model-complexity curves.

Read how Machine Learning System Design explains gradient boosting →

The Art of AI Product Development

A gradient boosting algorithm builds an ensemble model by sequentially adding weak predictive models that reduce errors made by the existing ensemble.

How it is used
Supports model selection by balancing forecast accuracy, runtime, computational cost, and client-defined error tolerance.
How deep it goes
Compares practical performance and runtime, but stops short of explaining boosting mechanics or implementation details.
Examples
Uses demand forecasts, a ±5% error margin, and runtime-versus-performance comparisons across models.
What you can do after
Evaluate whether gradient boosting satisfies a defined error threshold while reducing training and execution time.

Read how The Art of AI Product Development explains gradient boosting →

Also covered in

What to know first

Related concepts

Questions about gradient boosting

What is gradient boosting?

Gradient boosting is an ensemble-learning technique that sequentially adds predictive models fitted to reduce a loss by following its gradient.

Which Manning books explain gradient boosting?

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Ensemble Methods for Machine Learning, Grokking Machine Learning, Build AI Drug Discovery Pipelines, Transfer Learning for Natural Language Processing, Fight Fraud with Machine Learning, Machine Learning System Design and The Art of AI Product Development.

What should I know before learning gradient boosting?

Manning books that teach gradient boosting usually introduce random forest, decision tree, language model and linear regression first.

What is the difference between gradient boosting and Newton boosting?

Gradient boosting is an ensemble-learning technique that sequentially adds predictive models fitted to reduce a loss by following its gradient. Newton boosting is a boosting approach that uses first- and second-order loss information to construct successive learners.

What kinds of gradient boosting are there?

Kinds of gradient boosting include stochastic gradient boosting.

What is gradient boosting a kind of?

Gradient boosting is a kind of ensemble. An ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.