Boosting is an ensemble technique that trains models sequentially: each new learner focuses on errors made by the existing ensemble. Unlike parallel ensembles such as random forests, boosting uses this sequence to reduce bias on difficult cases while combining efficient weak learners. section 6.4.2
Gradient boosting passes error information forward through a loss function. Each iteration adjusts the next tree to improve the ensemble’s fit to the training examples, producing the XGBoost-style workflow used for molecular prediction. section 6.4.2
Drug-discovery team analogy
Imagine scientists screening compounds for binding to an RNA target. The first scientist makes initial predictions; later scientists study misclassified compounds and correct those mistakes. Their combined expertise becomes stronger on increasingly difficult candidates. section 6.4.2
Figure 6.13 Parallel ensembles train multiple independent models and average their predictions to reduce variance, while sequential ensembles like boosting train a sequence of models where each new model corrects errors made by the previous models, reducing bias.

Compare parallel ensembles, which average independent models to reduce variance, with boosting, which sequences corrective models to reduce bias. section 6.4.2




