liveBook · Manning

Early stopping

As explained in Build AI Drug Discovery Pipelines

Early stopping is a training-control technique that terminates iterative model fitting when additional updates no longer improve a selected evaluation criterion.

Early stopping controls iterative model training by watching performance on held-out validation data. When the metric stops improving, training ends instead of continuing toward overfitting. section 11.5.6 section 6.4.2

A practical early-stopping workflow is:

  • Reserve validation data and evaluate the model after each training epoch or boosting iteration. section 11.5.6 section 6.4.2
  • Track a chosen criterion, such as validation mean squared error, and retain the checkpoint with the best value. section 11.5.6
  • Stop after performance fails to improve for a configured number of evaluations, rather than using every permitted epoch or tree. section 11.5.6 section 6.4.2

Solubility regression

An SGDRegressor pipeline combines engineered molecular descriptors with polynomial features, scaling, and early_stopping=True. Its loss, learning-rate schedule, and stopping choice are tuned alongside other parameters before evaluation against ESOL and y-scrambled baselines. section 4.3.3

Neural-network affinity prediction

During five-fold cross-validation, a graph neural network trains for up to 2000 epochs, evaluates each validation fold after every epoch, saves the lowest-validation-MSE checkpoint, and stops when improvement stalls. section 11.5.6

Listing 4.9 Evaluate Best Performing Model against ESOL and y-Scrambled Baselines
from tqdm.notebook import trange
from sklearn.linear_model import SGDRegressor
engineered_features = list(RDKIT_DESCRIPTORS.keys())
X = pd.concat([X_train, X_val])
y = pd.concat([y_train, y_val])
 
sgd_regressor = make_pipeline(
  SimpleImputer(strategy="median"),
  PolynomialFeatures(degree=2),
  StandardScaler(),
  SGDRegressor(
      alpha=0.1, early_stopping=True, learning_rate='invscaling',
      loss="epsilon_insensitive", penalty='l2'
  ),
)
 
sgd_regressor.fit(X, y)
 
PandasTools.AddMoleculeColumnToFrame(test_df, 'Drug', 'Molecule')
test_df['fragment_count'] = test_df.Molecule.apply(
    num_fragments
)
test_df = test_df[test_df.fragment_count == 1]
for desc, func in RDKIT_DESCRIPTORS.items():
  test_df[desc] = test_df["Molecule"].apply(func)
X_test, y_test = test_df[engineered_features], test_df.Y
 
y_test_preds = sgd_regressor.predict(X_test)
test_rmse = np.sqrt(mean_squared_error(y_test, y_test_preds))
 
print(f"SGD Regressor (SVR) Test RMSE: {test_rmse:.4f}")
# Output -> SGD Regressor (SVR) Test RMSE: 1.2783
 
esol_test_pred_original = (
    0.16
    - 0.63 * test_df.logP
    - 0.0062 * test_df["Molecular Weight"]
    + 0.066 * test_df["Number of Rotatable Bonds"]
    - 0.74 * test_df["Aromatic Proportion"]
)
print(
  f"RMSE with original ESOL coefficients: "
  f"{np.sqrt(mean_squared_error(test_df.Y, esol_test_pred_original)):.4f}"
)
# Output -> RMSE with original ESOL coefficients: 1.3787
 
y_scrambled_rmse = []
num_iterations = 100
for i in trange(num_iterations):
  y_scrambled = y.sample(frac=1, random_state=None).to_numpy()
  sgd_regressor.fit(X, y_scrambled)
  y_pred = sgd_regressor.predict(X_test)
  y_scrambled_rmse.append(np.sqrt(mean_squared_error(y_test, y_pred)))
 
average_rmse = sum(y_scrambled_rmse) / num_iterations
print(
    f'Average RMSE over {num_iterations} y-scrambled iterations: '
    f'{average_rmse}'
)
# Output -> Average RMSE over 100 y-scrambled iterations: 2.4585

Inspect the runnable regression pipeline that enables early stopping and places it alongside loss, learning-rate, and regularization choices. section 4.3.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains validation checkpoints, epoch limits, and stalled-improvement stopping, but does not specify a universal patience value.
Examples
Uses molecular-solubility regression, graph neural networks for drug-target affinity, and XGBoost tree-count selection.
What it assumes
Assumes training and validation data, an evaluation metric, and iterative epochs or boosting iterations.
What you can do after
Operate/configure training with validation monitoring, best-checkpoint retention, and early termination.
How it connects
Connects validation sets, cross-validation, loss functions, learning rates, overfitting, and gradient boosting.

Also explained in

Compare how 11 books explain early stopping →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.