Weak learner
A weak learner is a predictive model that performs only modestly better than chance or a simple baseline and can contribute to a stronger ensemble.
Grokking Machine Learning
A weak learner is a predictive model that performs only modestly better than chance but can contribute to a strong learner when combined with other weak learners.
- How it is used
- Uses individual predictive models as components of ensemble methods for classification and regression.
- How deep it goes
- Works through random sampling, mistake-focused training, residual fitting, voting, and additive predictions using small decision trees.
- Examples
- Illustrates shallow trees on spam emails, weighted classifiers on two-class points, and residual-fitting trees on app engagement data.
- What you can do after
- Apply it via Scikit-Learn ensemble classifiers and regressors, then inspect how multiple trees contribute to predictions.
Ensemble Methods for Machine Learning
A weak learner is a simple predictive model that performs only slightly better than random guessing.
- How it is used
- Combines simple base estimators sequentially to build stronger boosting ensembles.
- How deep it goes
- Defines weak learners, contrasts them with strong learners, and traces their weighted training and combination in AdaBoost.
- Examples
- Uses depth-1 decision stumps, weighted misclassified points, a three-stump ensemble, and a make_moons classification data set.
- What you can do after
- Build it from scratch as a weighted AdaBoost ensemble of decision stumps and evaluate its prediction error.
Read how Ensemble Methods for Machine Learning explains weak learner →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A predictive model that performs only slightly better than a simple baseline and is combined with other models in an ensemble.
- How it is used
- Deploys simple predictive models as components of averaging, bagging, AdaBoost, and gradient-boosting ensembles.
- How deep it goes
- Explains weak learners through bias, variance, sequential correction, and weighting, but does not derive their mathematics.
- Examples
- Uses linear regression, k-nearest neighbors, decision trees, AdaBoost, and XGBoost’s n_estimators parameter as concrete examples.
- What you can do after
- Apply via XGBoost by balancing the number of weak learners, learning rate, predictive performance, and training time.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains weak learner →
Related concepts
- AdaBoostused by · AdaBoost is an ensemble-learning algorithm that sequentially combines weak learners, adaptively reweighting training examples so that later learners emphasize those earlier learners handled poorly.
- boostingused by · Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.
- decision treekind of · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.
- gradient boostingused by · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.
- sequential ensemble methodused by · A sequential ensemble method trains base learners in successive stages, with later learners depending on or correcting the outputs of earlier learners.
- baggingused by · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.