Boosting combines predictions from multiple limited models, trained sequentially. Each new model concentrates on observations earlier models handled poorly, correcting the ensemble step by step. section 8.8.3 section 9.2.3
A boosted classifier’s margin measures the combined vote’s strength for the correct class relative to the competing class. AdaBoost can continue increasing margins after training accuracy is perfect, offering one explanation for continued improvement on new data. This margin is not the geometric distance used by a support vector machine. section 8.8.3
Boosting and bagging build ensembles differently:
- Bagging trains models independently on bootstrap samples; boosting trains sequentially through successive correction. section 9.2.3
- AdaBoost, introduced in 1995, and later methods such as gradient boosting and XGBoost belong to the boosting family. section 9.2.3




