Regularization constrains learning to reduce overfitting: a model’s tendency to learn noise in its training data rather than patterns that generalize to unseen examples. In molecular prediction, this matters when a model must make reliable predictions for compounds it has never encountered. section 3.4.1 section 3.3.1
One approach adds a penalty for large coefficients to the cost function. Training then balances prediction error against coefficient size. The penalty strength λ controls this tradeoff: a higher value imposes a stronger constraint. section 3.4.1 section 3.5
Common coefficient penalties include:
- L1: Penalizes the sum of absolute weights and can drive unimportant weights to zero, effectively selecting features. section 3.4.1
- L2: Penalizes the sum of squared weights, encouraging smaller, more evenly distributed weights. section 3.4.1
- Elastic net: Combines L1 and L2 penalties. section 3.4.1
Predicting hERG blockage
Molecular fingerprints represent structures as bits used as model features. Because one structure can set multiple bits, features can be correlated, making coefficients unstable and difficult to interpret. L1 regularization is especially relevant to these sparse features because it can set some weights to zero. section 3.4.1
Regularization is not limited to coefficient penalties. For a decision tree, limiting depth or requiring more samples at a leaf can reduce variance, though this can increase bias. Pruning instead removes splits that do not add value, using validation error to assess them. section 5.3.1
Predicting drug–target affinity
A graph neural network uses dropout after fully connected layers: during training, it randomly zeroes elements of feature vectors to discourage reliance on particular features. Its graph convolution layers also share parameters across neighborhoods, reducing the effective number of parameters. section 11.5.2 section 11.5.6
Listing 11.7 Process Molecular Graph through GCN Layers
def forward_mol(self, data_mol):
mol_x, mol_edge_index, mol_batch = \
data_mol.x, data_mol.edge_index, data_mol.batch
x = self.relu(self.mol_conv1(mol_x, mol_edge_index))
x = self.relu(self.mol_conv2(x, mol_edge_index))
x = self.relu(self.mol_conv3(x, mol_edge_index))
x = global_mean_pool(x, mol_batch)
x = self.relu(self.mol_fc_g1(x))
x = self.dropout(x)
x = self.mol_fc_g2(x)
x = self.dropout(x)
return xThe molecular graph forward pass applies dropout after each fully connected layer. section 11.5.2
Choose regularization strength and other hyperparameters by evaluation rather than assuming more constraint is always better. Cross-validation uses training data for both training and evaluation; grid search and randomized search provide ways to explore hyperparameters. section 3.5
















