ScorecardMonitoring compares a scorecard’s distributions across a reference population, such as past applicants, and a current population, such as newer applicants. This helps teams detect changes that may affect model stability. section 6.1.4
A typical monitoring workflow is:
- Fit the monitor with the scorecard and the two populations. The example uses psi_method="cart" to generate bins from decision-tree splits. section 6.1.4
- Review psi_table() for reference-versus-current counts and PSI values by bin. The example’s total PSI is 0.000568, with every bin below 0.10; these values indicate negligible distribution change. section 6.1.4
- Use tests_table() to check for event-rate shifts by bin, then system_stability_report() to review PSI, p-values, and model-level performance comparisons together. section 6.1.4
Listing 6.8 Monitoring distribution stability
from optbinning.scorecard import ScorecardMonitoring
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y
)
# Re-fit the scorecard on the training subset
scorecard.fit(X_train, y_train)
monitoring = ScorecardMonitoring(
scorecard=scorecard,
psi_method="cart",
psi_n_bins=10,
verbose=True
)
monitoring.fit(X_test, y_test, X_train, y_train)
See a Python example that fits the scorecard and initializes ScorecardMonitoring with training and test populations. section 6.1.4
A simple train-test split demonstrates the workflow; in deployment, use populations from different time windows to reflect actual monitoring. PSI and significance tests provide complementary checks for distribution and event-rate changes. section 6.1.4 section 6.6
