liveBook · Manning

Dropout

As explained in Build AI Drug Discovery Pipelines

Dropout is a regularization technique that randomly disables units during training to reduce reliance on particular representations.

Dropout is a regularization technique that helps limit overfitting in neural networks. During training, it randomly sets a fraction of a layer’s inputs to zero, discouraging the network from relying too heavily on particular features or combinations of features. section 8.2.2 section 11.5.2

For example, nn.Dropout(0.2) randomly zeros 20% of its inputs during a training update. Forward and backward passes use the resulting network to update weights and biases; a different random subset is selected for the next batch. During inference, dropout is turned off and all neurons are active. section 8.2.2

Listing 8.6 Kinase Binder Neural Network Architecture
class KinaseBinderNN(nn.Module):
    def __init__(self, input_size=2048):
        super(KinaseBinderNN, self).__init__()
        self.fc1 = nn.Linear(input_size, 512)
        self.fc2 = nn.Linear(512, 256)
        self.fc3 = nn.Linear(256, 1)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.2)
        
    def forward(self, x):
        x = self.dropout(self.relu(self.fc1(x)))
        x = self.dropout(self.relu(self.fc2(x)))
        x = self.fc3(x)
        return x
 

This kinase-binder network applies PyTorch dropout after each hidden linear layer and ReLU activation, leaving the output layer without dropout. section 8.2.2

Dropout in a docking surrogate

A surrogate model predicts docking scores from molecular fingerprints during active learning. Its two hidden layers use dropout with probability 0.5 to help limit overfitting as the training data grows. Its prediction method calls self.eval() before inference. section 9.2.2

Dropout also regularizes a graph neural network for drug-target affinity: feature-vector elements are zeroed after fully connected layers. One configuration uses a dropout probability of 0.2. section 11.5.2 section 11.5.6

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains random zeroing across training batches and its removal during inference, with layer-level implementations.
Examples
Uses a kinase-binder network and a docking-score surrogate with dropout probabilities of 0.2 and 0.5, respectively.
What you can do after
Apply dropout via PyTorch nn.Dropout in hidden layers and switch to evaluation mode for prediction.
How it connects
Connects dropout to regularization, overfitting, active learning, and graph neural networks.

Also explained in

Compare how 16 books explain dropout →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.