Boosting
Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.
Grokking Machine Learning
An ensemble technique that trains learners sequentially so later learners focus on weaknesses of earlier learners and combines their predictions.
- How it is used
- Combines sequentially trained weak learners into a stronger model by directing later learners toward earlier mistakes.
- How deep it goes
- Explains AdaBoost’s reweighting and weighted voting with a worked classification example, without detailing gradient boosting or XGBoost.
- Examples
- Uses a triangle-and-square classification dataset and a spam-email example with “lottery” and “spam” features.
- What you can do after
- Apply it via Scikit-Learn’s AdaBoostClassifier to fit and score a spam classifier.
Build AI Drug Discovery Pipelines
Boosting is an ensemble technique that sequentially trains models so later models correct errors made by earlier models.
- How it is used
- Applies sequential ensembles to predict small-molecule binding to an RNA target.
- How deep it goes
- Explains weak learners, sequential error correction, and loss-driven gradient boosting updates.
- Examples
- Uses an RNA-target compound-screening analogy and XGBoost feature-importance metrics including gain, coverage, and frequency.
- What you can do after
- Apply via XGBoost to understand sequential model correction and aggregated predictions on difficult molecular cases.
Read how Build AI Drug Discovery Pipelines explains boosting →
Ensemble Methods for Machine Learning
Boosting is an ensemble technique that combines weak learners trained sequentially so that later learners improve on earlier mistakes.
- How it is used
- Frames boosting as a sequential ensemble technique for turning weak learners into a strong learner.
- How deep it goes
- Explains sequential dependence, weak learners, and adaptive reweighting, but stops short of full update equations.
- Examples
- Uses decision stumps, shallow decision trees, adaptive example weights, and the amateurs-versus-professionals analogy.
- What you can do after
- Build from scratch a sequential ensemble that uses weak learners and adaptive reweighting to correct earlier mistakes.
Read how Ensemble Methods for Machine Learning explains boosting →
Machine Learning Algorithms in Depth
Boosting is an ensemble learning technique that combines weak learners trained sequentially with greater emphasis on previously misclassified examples.
- How it is used
- Uses sequentially dependent learners to reduce bias and strengthen predictions in supervised classification and regression.
- How deep it goes
- Explains sequential weighting, weighted errors, and prediction combination, while stating gradient boosting’s loss minimization without fully deriving it.
- Examples
- Uses depth-one trees, Iris features, an XOR dataset, decision regions, learning curves, and varying ensemble sizes.
- What you can do after
- Apply it via scikit-learn’s AdaBoostClassifier and evaluate ensembles with different numbers of weak learners.
Read how Machine Learning Algorithms in Depth explains boosting →
Timeless Algorithms, The Foundational Ideas
Boosting is an ensemble technique that builds models sequentially, with later models focusing on cases earlier models handled poorly.
- How it is used
- Combines models through sequential correction, contrasting this ensemble approach with independent bagging.
- How deep it goes
- Defines sequential focus on earlier errors and interprets classifier margins, but does not detail training steps or equations.
- Examples
- Names AdaBoost, gradient boosting, and XGBoost as boosting methods, dating AdaBoost to 1995.
- What you can do after
- Understand how sequential correction differs from bagging and how boosting margins can grow after training accuracy is perfect.
Read how Timeless Algorithms, The Foundational Ideas explains boosting →
Also covered in
What to know first
- random forestA random forest is an ensemble learning model that aggregates predictions from multiple decision trees trained with randomized data samples, feature subsets, or both.
- AdaBoostAdaBoost is an ensemble-learning algorithm that sequentially combines weak learners, adaptively reweighting training examples so that later learners emphasize those earlier learners handled poorly.
- decision treeA decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
- ensembleAn ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.
Related concepts
- weak learneruses · A weak learner is a predictive model that performs only modestly better than chance or a simple baseline and can contribute to a stronger ensemble.
- AdaBoostincludes · AdaBoost is an ensemble-learning algorithm that sequentially combines weak learners, adaptively reweighting training examples so that later learners emphasize those earlier learners handled poorly.
- gradient boostingincludes · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models fitted to reduce a loss by following its gradient.
- ensemblekind of · An ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.
- baggingcontrasts with · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
- biasaddresses · Bias is a systematic tendency or deviation that distorts measurements, estimates, judgments, or outcomes relative to a relevant reference or target.
Questions about boosting
What is boosting?
Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.
Which Manning books explain boosting?
Grokking Machine Learning, Build AI Drug Discovery Pipelines, Ensemble Methods for Machine Learning, Machine Learning Algorithms in Depth and Timeless Algorithms, The Foundational Ideas.
What should I know before learning boosting?
Manning books that teach boosting usually introduce random forest, AdaBoost, decision tree and ensemble first.
What is the difference between boosting and bagging?
Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones. Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
What kinds of boosting are there?
Kinds of boosting include AdaBoost and gradient boosting.
What is boosting a kind of?
Boosting is a kind of ensemble. An ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




