liveBook · Manning

Stratified sampling

As explained in Build AI Drug Discovery Pipelines

Stratified sampling partitions data while preserving specified class or label distributions across the resulting subsets.

Stratified sampling divides a dataset into subsets while preserving label distributions. In multilabel drug classification, it keeps uncommon labels and label combinations represented across training and test data. section 5.4.2

Use stratification to:

  • Balance unevenly distributed inhibitor labels between training and test sets. section 5.4.2
  • Preserve correlations among multiple labels by retaining representative label combinations. section 5.4.2
  • Support more reliable generalization estimates on unseen molecules. section 5.4.2

CYP inhibitor classification

After retaining 4,805 molecules observed across five CYP inhibitor datasets, stratified sampling creates training and test subsets that represent the five-label distribution and its combinations. section 5.4.2

Listing 5.10 Create Multilabel CYP Inhibitor Data and Train and Evaluate Models
from rdkit.Chem import MolToInchi, MolFromInchi
from skmultilearn.model_selection import IterativeStratification
from sklearn.multioutput import MultiOutputClassifier
from sklearn.metrics import coverage_error, label_ranking_loss
 
def process_cyp_data(cyp):
  df = ADME(name = f'{cyp}_Veith').get_data()
  df = df.rename(columns={'Y':f'{cyp}'})
  PandasTools.AddMoleculeColumnToFrame(df, "Drug", "Molecule")
  df['Inchi'] = df.Molecule.apply(MolToInchi)
  df = df.dropna()
  return df
 
cyps = ['CYP2C19', 'CYP2D6', 'CYP3A4', 'CYP1A2', 'CYP2C9']
cyp_inhibitor_dfs = [process_cyp_data(cyp) for cyp in cyps]
cyp_inhibitor_df = cyp_inhibitor_dfs[0]
for df in cyp_inhibitor_dfs[1:]:
  cyp_inhibitor_df = cyp_inhibitor_df.merge(df, on='Inchi', how='inner')
 
cyp_inhibitor_df = cyp_inhibitor_df[
    ['Molecule', 'CYP2C19', 'CYP2D6', 'CYP3A4', 'CYP1A2', 'CYP2C9']
]
cyp_inhibitor_df['fingerprints'] = cyp_inhibitor_df['Molecule'].apply(
    compute_fingerprint, r=2, nBits=2048
)
 
X = cyp_inhibitor_df["fingerprints"]
y = cyp_inhibitor_df[cyps]
 
stratifier = IterativeStratification(
    n_splits=2, order=2, sample_distribution_per_fold=[0.2, 0.8]
)
train_indexes, test_indexes = next(stratifier.split(X, y))
X_train, y_train = X.loc[train_indexes], y.loc[train_indexes]
X_test, y_test = X.loc[test_indexes], y.loc[test_indexes]
 
lr_clf = MultiOutputClassifier(LogisticRegression()).fit(
    np.array(X_train.values.tolist()), y_train
)
lr_y_pred = lr_clf.predict(np.array(X_test.values.tolist()))
lr_y_pred_proba = lr_clf.predict_proba(np.array(X_test.values.tolist()))
 
rf_clf = RandomForestClassifier(n_estimators=500, n_jobs=-1).fit(
    np.array(X_train.values.tolist()), y_train
)
rf_y_pred = rf_clf.predict(np.array(X_test.values.tolist()))
rf_y_pred_proba = rf_clf.predict_proba(np.array(X_test.values.tolist()))
 
print(f"LR Coverage Error: {coverage_error(y_test, lr_y_pred):.3f}")
# Output -> LR Coverage Error: 2.863
print(f"RF Coverage Error: {coverage_error(y_test, rf_y_pred):.3f}")
# Output -> RF Coverage Error: 2.862
print(f"LR Ranking Loss: {label_ranking_loss(y_test, lr_y_pred):.3f}")
# Output -> LR Ranking Loss: 0.481
print(f"RF Ranking Loss: {label_ranking_loss(y_test, rf_y_pred):.3f}")
# Output -> RF Ranking Loss: 0.476
 

The listing demonstrates a multilabel workflow using scikit-multilearn's IterativeStratification alongside molecular data preparation and model evaluation. section 5.4.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains label preservation and demonstrates implementation, but stops short of mathematical derivation.
Examples
Uses five CYP inhibitor labels, 4,805 shared molecules, and IterativeStratification.
What it assumes
Assumes multilabel classification terminology and teaches why uneven labels and correlated combinations require careful splitting.
What you can do after
Apply it via IterativeStratification to construct representative multilabel training and test sets.
How it connects
Connects multilabel classification, label dependencies, generalization, and negative sampling.

Also explained in

Compare how 4 books explain stratified sampling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.