A hidden layer is an intermediate layer of artificial neurons between a neural network’s input and output layers. Each neuron applies a weighted calculation and an activation function; its output becomes a feature for a subsequent layer. section 8.2.1 section 8.3
Hidden layers with nonlinear activation functions let a network capture nonlinear relationships. By contrast, a single logistic regression unit has a linear decision boundary in the original feature space. Network depth refers to the number of layers; width refers to the number of neurons per layer. section 8.2.1
Kinase binder network
A PyTorch network takes 2,048 inputs through hidden layers of 512 and 256 neurons, then produces one output. Each hidden layer applies a linear transformation, ReLU activation, and dropout at a rate of 0.2. Its first weight matrix has shape 512 × 2,048. section 8.2.2
Listing 8.6 Kinase Binder Neural Network Architecture
class KinaseBinderNN(nn.Module):
def __init__(self, input_size=2048):
super(KinaseBinderNN, self).__init__()
self.fc1 = nn.Linear(input_size, 512)
self.fc2 = nn.Linear(512, 256)
self.fc3 = nn.Linear(256, 1)
self.relu = nn.ReLU()
self.dropout = nn.Dropout(0.2)
def forward(self, x):
x = self.dropout(self.relu(self.fc1(x)))
x = self.dropout(self.relu(self.fc2(x)))
x = self.fc3(x)
return x
The listing shows how the two hidden layers and their activations are chained in the forward method. section 8.2.2
Weights are initialized to small random values so neurons in the same layer can learn different features. If all weights start at the same constant, those neurons receive the same gradients and update in the same way. section 8.2.2










