AdaBoost
AdaBoost is an ensemble-learning algorithm that sequentially combines weak learners, adaptively reweighting training examples so that later learners emphasize those earlier learners handled poorly.
Ensemble Methods for Machine Learning
AdaBoost is an adaptive boosting technique that combines sequentially trained weak learners while increasing emphasis on previously misclassified examples.
- How it is used
- Builds a sequential ensemble by directing later weak learners toward earlier classification mistakes.
- How deep it goes
- Works through weighted fitting, error-based learner weighting, prediction, scikit-learn usage, tuning, and outlier limitations without formal proofs.
- Examples
- Uses weighted decision stumps, make_moons, breast cancer and iris classification, and handwritten-digit classification.
- What you can do after
- Build it from scratch or apply it through AdaBoostClassifier, then tune estimator count and learning rate with cross-validation.
Read how Ensemble Methods for Machine Learning explains AdaBoost →
Grokking Machine Learning
An adaptive boosting algorithm that combines sequentially trained weak learners by weighting their contributions according to predictive performance.
- How it is used
- Builds a strong ensemble by training weak learners sequentially, directing each one toward examples earlier learners misclassified.
- How deep it goes
- Explains sequential reweighting, depth-1 trees, log-odds voting, and a classification walkthrough; mentions regression without developing that case.
- Examples
- Uses triangle-and-square datasets, spam features, weighted-vote exercises, and a Scikit-Learn classifier example.
- What you can do after
- Apply it via Scikit-Learn to train an AdaBoost classifier and score its fit on labeled data.
Also covered in
What to know first
- decision treeA decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
- ensembleAn ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.
- overfittingOverfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
- classifierA classifier is a predictive model that assigns an input to one or more discrete classes or categories.
Related concepts
- weak learneruses · A weak learner is a predictive model that performs only modestly better than chance or a simple baseline and can contribute to a stronger ensemble.
- exponential lossuses · Exponential loss is a classification loss that increases exponentially as an example’s prediction margin becomes more incorrect.
- reweightinguses · Reweighting is a technique that changes the relative influence of examples, groups, or outcomes by assigning them different weights during learning or analysis.
- sequential ensemble methodkind of · A sequential ensemble method trains base learners in successive stages, with later learners depending on or correcting the outputs of earlier learners.
- boostingkind of · Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.
- decision stumpuses · A decision stump is a decision tree of depth one with a single decision node and prediction leaves.
Questions about AdaBoost
What is AdaBoost?
AdaBoost is an ensemble-learning algorithm that sequentially combines weak learners, adaptively reweighting training examples so that later learners emphasize those earlier learners handled poorly.
Which Manning books explain AdaBoost?
Ensemble Methods for Machine Learning and Grokking Machine Learning.
What should I know before learning AdaBoost?
Manning books that teach AdaBoost usually introduce decision tree, ensemble, overfitting and classifier first.
What is AdaBoost a kind of?
AdaBoost is a kind of sequential ensemble method. A sequential ensemble method trains base learners in successive stages, with later learners depending on or correcting the outputs of earlier learners.
What is AdaBoost also called?
AdaBoost is also called adaptive boosting.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

