liveBook · Manning

Boosting

Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.

How 5 Manning books explain it

Grokking Machine Learning

An ensemble technique that trains learners sequentially so later learners focus on weaknesses of earlier learners and combines their predictions.

How it is used
Combines sequentially trained weak learners into a stronger model by directing later learners toward earlier mistakes.
How deep it goes
Explains AdaBoost’s reweighting and weighted voting with a worked classification example, without detailing gradient boosting or XGBoost.
Examples
Uses a triangle-and-square classification dataset and a spam-email example with “lottery” and “spam” features.
What you can do after
Apply it via Scikit-Learn’s AdaBoostClassifier to fit and score a spam classifier.

Read how Grokking Machine Learning explains boosting →

Build AI Drug Discovery Pipelines

Boosting is an ensemble technique that sequentially trains models so later models correct errors made by earlier models.

How it is used
Applies sequential ensembles to predict small-molecule binding to an RNA target.
How deep it goes
Explains weak learners, sequential error correction, and loss-driven gradient boosting updates.
Examples
Uses an RNA-target compound-screening analogy and XGBoost feature-importance metrics including gain, coverage, and frequency.
What you can do after
Apply via XGBoost to understand sequential model correction and aggregated predictions on difficult molecular cases.

Read how Build AI Drug Discovery Pipelines explains boosting →

Ensemble Methods for Machine Learning

Boosting is an ensemble technique that combines weak learners trained sequentially so that later learners improve on earlier mistakes.

How it is used
Frames boosting as a sequential ensemble technique for turning weak learners into a strong learner.
How deep it goes
Explains sequential dependence, weak learners, and adaptive reweighting, but stops short of full update equations.
Examples
Uses decision stumps, shallow decision trees, adaptive example weights, and the amateurs-versus-professionals analogy.
What you can do after
Build from scratch a sequential ensemble that uses weak learners and adaptive reweighting to correct earlier mistakes.

Read how Ensemble Methods for Machine Learning explains boosting →

Machine Learning Algorithms in Depth

Boosting is an ensemble learning technique that combines weak learners trained sequentially with greater emphasis on previously misclassified examples.

How it is used
Uses sequentially dependent learners to reduce bias and strengthen predictions in supervised classification and regression.
How deep it goes
Explains sequential weighting, weighted errors, and prediction combination, while stating gradient boosting’s loss minimization without fully deriving it.
Examples
Uses depth-one trees, Iris features, an XOR dataset, decision regions, learning curves, and varying ensemble sizes.
What you can do after
Apply it via scikit-learn’s AdaBoostClassifier and evaluate ensembles with different numbers of weak learners.

Read how Machine Learning Algorithms in Depth explains boosting →

Timeless Algorithms, The Foundational Ideas

Boosting is an ensemble technique that builds models sequentially, with later models focusing on cases earlier models handled poorly.

How it is used
Combines models through sequential correction, contrasting this ensemble approach with independent bagging.
How deep it goes
Defines sequential focus on earlier errors and interprets classifier margins, but does not detail training steps or equations.
Examples
Names AdaBoost, gradient boosting, and XGBoost as boosting methods, dating AdaBoost to 1995.
What you can do after
Understand how sequential correction differs from bagging and how boosting margins can grow after training accuracy is perfect.

Read how Timeless Algorithms, The Foundational Ideas explains boosting →

Also covered in

What to know first

Related concepts

Questions about boosting

What is boosting?

Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.

Which Manning books explain boosting?

Grokking Machine Learning, Build AI Drug Discovery Pipelines, Ensemble Methods for Machine Learning, Machine Learning Algorithms in Depth and Timeless Algorithms, The Foundational Ideas.

What should I know before learning boosting?

Manning books that teach boosting usually introduce random forest, AdaBoost, decision tree and ensemble first.

What is the difference between boosting and bagging?

Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones. Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.

What kinds of boosting are there?

Kinds of boosting include AdaBoost and gradient boosting.

What is boosting a kind of?

Boosting is a kind of ensemble. An ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.