Boosting
Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.
Grokking Machine Learning
An ensemble technique that trains learners sequentially so later learners focus on weaknesses of earlier learners and combines their predictions.
- How it is used
- Combines sequentially trained weak learners into a stronger model by directing later learners toward earlier mistakes.
- How deep it goes
- Explains AdaBoost’s reweighting and weighted voting with a worked classification example, without detailing gradient boosting or XGBoost.
- Examples
- Uses a triangle-and-square classification dataset and a spam-email example with “lottery” and “spam” features.
- What you can do after
- Apply it via Scikit-Learn’s AdaBoostClassifier to fit and score a spam classifier.
Ensemble Methods for Machine Learning
Boosting is an ensemble technique that combines weak learners trained sequentially so that later learners improve on earlier mistakes.
- How it is used
- Frames boosting as a sequential ensemble technique for turning weak learners into a strong learner.
- How deep it goes
- Explains sequential dependence, weak learners, and adaptive reweighting, but stops short of full update equations.
- Examples
- Uses decision stumps, shallow decision trees, adaptive example weights, and the amateurs-versus-professionals analogy.
- What you can do after
- Build from scratch a sequential ensemble that uses weak learners and adaptive reweighting to correct earlier mistakes.
Read how Ensemble Methods for Machine Learning explains boosting →
Machine Learning Algorithms in Depth
Boosting is an ensemble learning technique that combines weak learners trained sequentially with greater emphasis on previously misclassified examples.
- How it is used
- Uses sequentially dependent learners to reduce bias and strengthen predictions in supervised classification and regression.
- How deep it goes
- Explains sequential weighting, weighted errors, and prediction combination, while stating gradient boosting’s loss minimization without fully deriving it.
- Examples
- Uses depth-one trees, Iris features, an XOR dataset, decision regions, learning curves, and varying ensemble sizes.
- What you can do after
- Apply it via scikit-learn’s AdaBoostClassifier and evaluate ensembles with different numbers of weak learners.
Read how Machine Learning Algorithms in Depth explains boosting →
Related concepts
- weak learneruses · A weak learner is a predictive model that performs only modestly better than chance or a simple baseline and can contribute to a stronger ensemble.
- gradient boostingincludes · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.
- AdaBoostincludes · AdaBoost is an ensemble-learning algorithm that sequentially combines weak learners, adaptively reweighting training examples so that later learners emphasize those earlier learners handled poorly.
- ensemblekind of · An ensemble is a technique that combines or considers outputs from multiple models or component methods collectively to produce or assess a joint result.
- baggingcontrasts with · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
- strong learnerproduces · A strong learner is a predictive model with substantially improved task performance, often formed by combining multiple weak learners.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.