liveBook · Manning

Hyperparameter tuning

As explained in Build AI Drug Discovery Pipelines

Hyperparameter tuning is the systematic selection of model configuration values that are not learned directly from the training examples.

Hyperparameter tuning selects configuration values that are not learned directly from training examples. The goal is to find a model that captures useful patterns without overfitting, so it can generalize to new molecules. Choices can affect featurization, the learning algorithm, and the model itself. section 3.4.3

For cardiotoxicity prediction, choices include the radius and bit count of Morgan fingerprints, the regularization technique and strength, and the degree of a polynomial transformation. Each additional choice increases the number of combinations to evaluate. section 3.0 section 3.4.3

A practical search workflow is:

  • Build a pipeline that combines preprocessing and featurization transformers with the final estimator. Keeping these steps together streamlines cross-validation and helps prevent data leakage. section 3.4.3
  • Choose candidate values and a score, then use cross-validation on the training data to compare configurations. section 3.4.3
  • Use grid search to evaluate every specified combination when the search space is small. Use randomized search to sample configurations when broader exploration is more practical. section 3.5 section 3.4.3

Grid search for a molecular classifier

A pipeline standardizes SMILES strings, generates fingerprints, and trains an SGDClassifier. GridSearchCV compares fingerprint radii of 2 and 4, bit counts of 1024 and 2048, L1 and L2 penalties, and three regularization strengths using five-fold cross-validation and F1 scoring. The best reported combination uses radius 2, 2048 bits, L2, and alpha 0.1, with a cross-validated F1 score of approximately 0.881. This is a search result, not a test-set score. section 3.4.3

Listing 3.14 Pipeline with GridSearchCV
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV
from sklearn.pipeline import make_pipeline, Pipeline
from sklearn.preprocessing import PolynomialFeatures
 
cache_dir = "gs_linear_sgd"
pipe_linear_sgd = make_pipeline(
  SmilesToMols(),  # Transformer: Load & standardize SMILES
  FingerprintFeaturizer(),  # Transformer: Feature generation step
  SGDClassifier(),  # Estimator: linear model via SGD
  memory=cache_dir
)
 
param_grid_linear_sgd = [{
  'sgdclassifier__penalty': ["l2", "l1"],
  'sgdclassifier__alpha': [1e-3, 1e-2, 1e-1],
  'fingerprintfeaturizer__radius': [2, 4],
  'fingerprintfeaturizer__nBits': [1024, 2048],
}]
 
gs_linear_sgd = GridSearchCV(
  estimator=pipe_linear_sgd,
  param_grid=param_grid_linear_sgd,
  scoring="f1",
  cv=5,
  verbose=1,
)
 
gs_linear_sgd.fit(train_set.SMILES, train_set.Class)

See how pipeline parameters are named and passed to GridSearchCV. section 3.4.3

Listing 3.16 Pipeline with RandomizedSearchCV and PolynomialDegree Transformer
from scipy.stats import uniform as sp_rand
 
pipe_nonlinear_sgd = make_pipeline(
  FingerprintFeaturizer(),
  PolynomialFeatures(degree=2, include_bias=False, interaction_only=True),
  SGDClassifier(max_iter=5000, early_stopping=True),
  memory="rs_nonlinear_sgd"
)
 
param_dist_nonlinear_sgd = [{
  'sgdclassifier__penalty': ["l1", "l2"],
  'sgdclassifier__alpha': sp_rand(),
  'fingerprintfeaturizer__radius': [2, 4],
  'fingerprintfeaturizer__nBits': [64, 128, 256],
}]
 
rs_nonlinear_sgd = RandomizedSearchCV(
  estimator=pipe_nonlinear_sgd,
  param_distributions=param_dist_nonlinear_sgd,
  scoring="f1",
  cv=5,
  n_iter=20,
)
rs_nonlinear_sgd.fit(train_set.mol, train_set.Class)
>>> rs_nonlinear_sgd.best_params_
{'fingerprintfeaturizer__nBits': 128, 'fingerprintfeaturizer__radius': 2, 'sgdclassifier__alpha': 0.36469724945730586, 'sgdclassifier__penalty': 'l2'}
 
>>> rs_nonlinear_sgd.best_score_
0.6930966168199139

Compare a randomized search that samples regularization strength and fingerprint settings while using polynomial features. section 3.4.3

Training curves can also guide adjustments. If validation loss levels off while training loss continues to fall, consider fewer epochs or early stopping, or adjust the learning rate or scheduler. section 8.2.3

In active learning, parameter sweeps can compare experimental configurations rather than just select a classifier. Varying initial sample size and acquisition strategy, for example, helps reveal sensitivity and interactions; consistent configuration supports fair comparison. section 9.2.7

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Works through pipeline construction, search spaces, cross-validation, and search results for grid and randomized search.
Examples
Uses cardiotoxicity classification with Morgan fingerprints and an SGD classifier, plus neural-network training and validation curves.
What you can do after
Apply it via Scikit-Learn pipelines, GridSearchCV, and RandomizedSearchCV to compare molecular model configurations.
How it connects
Connects tuning to featurization, regularization, cross-validation, overfitting, and early stopping.

Also explained in

Compare how 22 books explain hyperparameter tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.