The Brier score measures probabilistic prediction error as the mean squared error between predicted probabilities and binary observed outcomes. For molecule-level inhibition predictions, it evaluates whether probability estimates align with what actually happened, complementing threshold-based classification metrics. section 5.1.3
Cytochrome P450 inhibition
A classifier predicting a 20% inhibition probability should produce inhibitors in roughly 20% of comparable cases. The Brier score summarizes squared deviations between these probabilities and the observed inhibitor outcomes across the validation set. section 5.1.3
Use the equation to calculate the average squared difference between each predicted probability Pᵢ and binary outcome Oᵢ across N observations. section 5.1.3
Interpret the score alongside calibration diagnostics:
- Compare probability estimates with observed frequencies using reliability diagrams. section 5.1.3
- Use Brier score with RMSCE, precision-recall curves, and ROC curves rather than relying only on a decision threshold. section 5.5
- Assess calibrated and uncalibrated predictions on validation or test data, including after Platt scaling or isotonic regression. section 5.1.3
