liveBook · Manning

Gradient boosting

As explained in Build AI Drug Discovery Pipelines

Gradient boosting is an ensemble learning technique that sequentially adds weak models trained to reduce the current ensemble's loss or errors.

Gradient boosting is a sequential ensemble technique that adds weak decision trees, each correcting errors made by the current ensemble. A loss function measures those errors, while residuals approximate its gradient and guide later updates. section 6.4.2

A decision-tree gradient-boosting workflow proceeds by:

  • Initializing a constant model that produces a starting prediction. section 6.4.2
  • Computing each training example’s residual from the current prediction and true label. section 6.4.2
  • Training a weak decision tree on the inputs paired with their residuals. section 6.4.2
  • Adding successive small updates so the ensemble improves on difficult or previously misclassified examples. section 6.4.2

RNA-binding small-molecule classification

An XGBClassifier was trained to distinguish 2,003 RNA-binding compounds from 77,678 protein-binding compounds using 1,670 Mordred molecular descriptors. Stratified 10-fold cross-validation evaluated area under the precision-recall curve (AUPRC); the untuned model achieved an average AUPRC of 0.79, compared with 0.37 for LASSO and 0.78 for a neural network. section 6.4.1

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains weak learners, residual-based updates, and the gradient rationale, then demonstrates XGBClassifier without detailing hyperparameter tuning.
Examples
Uses RNA-binding versus protein-binding compounds, Mordred descriptors, XGBClassifier, AUPRC, and feature importance by gain, coverage, and frequency.
What it assumes
Assumes molecular descriptors, QSAR modeling, classification, and cross-validation, while defining boosting and residual correction incrementally.
What you can do after
Apply it via XGBClassifier to classify molecular compounds with stratified cross-validation and precision-recall evaluation.
How it connects
Connects decision trees, weak learners, loss functions, residuals, QSAR, molecular descriptors, cross-validation, and neural networks.

Also explained in

Compare how 8 books explain gradient boosting →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.