AdaBoost
AdaBoost is an ensemble-learning algorithm that sequentially combines weak learners, adaptively reweighting training examples so that later learners emphasize those earlier learners handled poorly.
Ensemble Methods for Machine Learning
AdaBoost is an adaptive boosting technique that combines sequentially trained weak learners while increasing emphasis on previously misclassified examples.
- How it is used
- Builds a sequential ensemble by directing later weak learners toward earlier classification mistakes.
- How deep it goes
- Works through weighted fitting, error-based learner weighting, prediction, scikit-learn usage, tuning, and outlier limitations without formal proofs.
- Examples
- Uses weighted decision stumps, make_moons, breast cancer and iris classification, and handwritten-digit classification.
- What you can do after
- Build it from scratch or apply it through AdaBoostClassifier, then tune estimator count and learning rate with cross-validation.
Read how Ensemble Methods for Machine Learning explains AdaBoost →
Grokking Machine Learning
An adaptive boosting algorithm that combines sequentially trained weak learners by weighting their contributions according to predictive performance.
- How it is used
- Builds a strong ensemble by training weak learners sequentially, directing each one toward examples earlier learners misclassified.
- How deep it goes
- Explains sequential reweighting, depth-1 trees, log-odds voting, and a classification walkthrough; mentions regression without developing that case.
- Examples
- Uses triangle-and-square datasets, spam features, weighted-vote exercises, and a Scikit-Learn classifier example.
- What you can do after
- Apply it via Scikit-Learn to train an AdaBoost classifier and score its fit on labeled data.
Related concepts
- weak learneruses · A weak learner is a predictive model that performs only modestly better than chance or a simple baseline and can contribute to a stronger ensemble.
- exponential lossuses · Exponential loss is a classification loss that increases exponentially as an example’s prediction margin becomes more incorrect.
- reweightinguses · Reweighting is a technique that changes the relative influence of examples, groups, or outcomes by assigning them different weights during learning or analysis.
- sequential ensemble methodkind of · A sequential ensemble method trains base learners in successive stages, with later learners depending on or correcting the outputs of earlier learners.
- boostingkind of · Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.
- decision stumpuses · A decision stump is a decision tree of depth one with a single decision node and prediction leaves.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.