A weak learner is an individually limited predictive model that is easy and efficient to train. Boosting combines many weak learners into a stronger ensemble: each new learner focuses on errors left by earlier learners. section 6.4.2
Boosting builds collective performance through an iterative workflow:
- Train an initial model to make predictions. section 6.4.2
- Measure prediction errors with a loss function. section 6.4.2
- Train the next learner on the problematic examples, using residuals as an error signal. section 6.4.2
- Aggregate the learners so the ensemble improves on difficult cases while reducing bias. section 6.4.2
Small-molecule binding prediction
For predicting whether small molecules bind an RNA target, an XGBoost ensemble trains successive tree-based learners. Each learner addresses errors from the previous model, helping the combined predictor identify difficult compounds. section 6.4.2
Figure 6.13 Parallel ensembles train multiple independent models and average their predictions to reduce variance, while sequential ensembles like boosting train a sequence of models where each new model corrects errors made by the previous models, reducing bias.

Compare parallel ensembles, which average independently trained models to reduce variance, with sequential boosting ensembles, which correct earlier errors to reduce bias. section 6.4.2



