Gradient boosting is a sequential ensemble technique that adds weak decision trees, each correcting errors made by the current ensemble. A loss function measures those errors, while residuals approximate its gradient and guide later updates. section 6.4.2
A decision-tree gradient-boosting workflow proceeds by:
- Initializing a constant model that produces a starting prediction. section 6.4.2
- Computing each training example’s residual from the current prediction and true label. section 6.4.2
- Training a weak decision tree on the inputs paired with their residuals. section 6.4.2
- Adding successive small updates so the ensemble improves on difficult or previously misclassified examples. section 6.4.2
RNA-binding small-molecule classification
An XGBClassifier was trained to distinguish 2,003 RNA-binding compounds from 77,678 protein-binding compounds using 1,670 Mordred molecular descriptors. Stratified 10-fold cross-validation evaluated area under the precision-recall curve (AUPRC); the untuned model achieved an average AUPRC of 0.79, compared with 0.37 for LASSO and 0.78 for a neural network. section 6.4.1







