liveBook · Manning

Platt scaling

As explained in Build AI Drug Discovery Pipelines

Platt scaling is a parametric probability-calibration method that fits a logistic regression mapping from model scores to outcome probabilities.

Platt scaling calibrates an already trained classifier by fitting logistic regression to its prediction scores. The calibrator converts those scores into probabilities that better reflect observed outcome frequencies, supporting confidence-based decisions and comparisons across models. section 5.1.3

A practical calibration workflow is:

  • Keep the original classifier fixed and generate its scores for validation instances. section 5.1.3
  • Fit logistic regression on those scores using the true validation labels as targets. section 5.1.3
  • Apply the fitted calibrator to held-out test scores and inspect predicted probabilities with reliability diagrams and calibration metrics. section 5.1.3
  • Prefer the sigmoid mapping when a parametric S-shaped relationship is suitable; compare isotonic regression when a flexible monotonic mapping is warranted. section 5.1.3
Listing 5.2 Calibration with Platt Scaling & Isotonic Regression
from sklearn.calibration import CalibratedClassifierCV
from sklearn.frozen import FrozenEstimator
from utils import reliability_diagram
 
lr_ps_cal = CalibratedClassifierCV(
    FrozenEstimator(lr_model), method='sigmoid'
)
lr_ps_cal.fit(val_fingerprints, val_df.Inhibitor)
test_proba_ps = lr_ps_cal.predict_proba(test_fingerprints)[:, 1]
ax, ax1 = reliability_diagram(test_proba_ps, test_df.Inhibitor, nbins=10)
 
lr_ir_cal = CalibratedClassifierCV(
    FrozenEstimator(lr_model), method='isotonic'
)
lr_ir_cal.fit(val_fingerprints, val_df.Inhibitor)
test_proba_ir = lr_ir_cal.predict_proba(test_fingerprints)[:, 1]
ax, ax1 = reliability_diagram(test_proba_ir, test_df.Inhibitor, nbins=10)

Use the scikit-learn listing to freeze a logistic model, fit Platt scaling with CalibratedClassifierCV, generate test probabilities, and compare them with isotonic calibration. section 5.1.3

CYP3A4 inhibition

For a CYP3A4 inhibitor classifier, fit the sigmoid calibrator on validation fingerprints and inhibitor labels, then evaluate test probabilities. Reliability diagrams, RMSCE, and Brier score show whether predicted confidence agrees with observed inhibition rates. section 5.1.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains score transformation, validation fitting, held-out testing, and sigmoid-versus-isotonic tradeoffs without deriving logistic-regression internals.
Examples
Uses CYP3A4 inhibition, molecular fingerprints, CalibratedClassifierCV, reliability diagrams, RMSCE, and Brier score.
What it assumes
Assumes a trained binary classifier, prediction scores, true labels, and separate validation and held-out test data.
What you can do after
Apply Platt scaling with CalibratedClassifierCV and evaluate calibrated test probabilities using reliability diagrams and calibration metrics.
How it connects
Connects probability calibration with logistic regression, isotonic regression, reliability diagrams, RMSCE, and Brier score.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.