Platt scaling calibrates an already trained classifier by fitting logistic regression to its prediction scores. The calibrator converts those scores into probabilities that better reflect observed outcome frequencies, supporting confidence-based decisions and comparisons across models. section 5.1.3
A practical calibration workflow is:
- Keep the original classifier fixed and generate its scores for validation instances. section 5.1.3
- Fit logistic regression on those scores using the true validation labels as targets. section 5.1.3
- Apply the fitted calibrator to held-out test scores and inspect predicted probabilities with reliability diagrams and calibration metrics. section 5.1.3
- Prefer the sigmoid mapping when a parametric S-shaped relationship is suitable; compare isotonic regression when a flexible monotonic mapping is warranted. section 5.1.3
Listing 5.2 Calibration with Platt Scaling & Isotonic Regression
from sklearn.calibration import CalibratedClassifierCV
from sklearn.frozen import FrozenEstimator
from utils import reliability_diagram
lr_ps_cal = CalibratedClassifierCV(
FrozenEstimator(lr_model), method='sigmoid'
)
lr_ps_cal.fit(val_fingerprints, val_df.Inhibitor)
test_proba_ps = lr_ps_cal.predict_proba(test_fingerprints)[:, 1]
ax, ax1 = reliability_diagram(test_proba_ps, test_df.Inhibitor, nbins=10)
lr_ir_cal = CalibratedClassifierCV(
FrozenEstimator(lr_model), method='isotonic'
)
lr_ir_cal.fit(val_fingerprints, val_df.Inhibitor)
test_proba_ir = lr_ir_cal.predict_proba(test_fingerprints)[:, 1]
ax, ax1 = reliability_diagram(test_proba_ir, test_df.Inhibitor, nbins=10)Use the scikit-learn listing to freeze a logistic model, fit Platt scaling with CalibratedClassifierCV, generate test probabilities, and compare them with isotonic calibration. section 5.1.3
CYP3A4 inhibition
For a CYP3A4 inhibitor classifier, fit the sigmoid calibrator on validation fingerprints and inhibitor labels, then evaluate test probabilities. Reliability diagrams, RMSCE, and Brier score show whether predicted confidence agrees with observed inhibition rates. section 5.1.3
