Isotonic regression calibrates an already trained classifier by mapping predicted scores to probabilities with a monotonic, piecewise-constant function. Unlike Platt scaling, it does not assume a sigmoid-shaped relationship, making it flexible for calibration tasks such as CYP inhibition prediction. section 5.1.3
Use isotonic regression when calibrated confidence matters for downstream decisions, ranking, or risk assessment:
- Fit the calibrator on validation-set scores and true labels rather than retraining the original classifier. section 5.1.3
- Prefer its flexibility when the score-to-outcome relationship may not follow a sigmoid curve. section 5.1.3
- Watch for outlier sensitivity, overfitting, and empty probability bins that leave estimates poorly distributed across 0 to 1. section 5.1.3
- Judge calibration with reliability diagrams and metrics such as RMSCE and Brier score, not with a single metric alone. section 5.1.3 section 5.5
Calibrating a CYP inhibition classifier
Freeze the trained logistic-regression model, fit CalibratedClassifierCV with method='isotonic' on validation fingerprints and inhibitor labels, then obtain test probabilities with predict_proba. Plot a ten-bin reliability diagram and compare the result with sigmoid calibration. section 5.1.3

