liveBook · Manning

Hidden layer

As explained in Build AI Drug Discovery Pipelines

A hidden layer is an intermediate neural-network layer whose artificial neurons transform representations between the input and output layers.

A hidden layer is an intermediate layer of artificial neurons between a neural network’s input and output layers. Each neuron applies a weighted calculation and an activation function; its output becomes a feature for a subsequent layer. section 8.2.1 section 8.3

Hidden layers with nonlinear activation functions let a network capture nonlinear relationships. By contrast, a single logistic regression unit has a linear decision boundary in the original feature space. Network depth refers to the number of layers; width refers to the number of neurons per layer. section 8.2.1

Kinase binder network

A PyTorch network takes 2,048 inputs through hidden layers of 512 and 256 neurons, then produces one output. Each hidden layer applies a linear transformation, ReLU activation, and dropout at a rate of 0.2. Its first weight matrix has shape 512 × 2,048. section 8.2.2

Listing 8.6 Kinase Binder Neural Network Architecture
class KinaseBinderNN(nn.Module):
    def __init__(self, input_size=2048):
        super(KinaseBinderNN, self).__init__()
        self.fc1 = nn.Linear(input_size, 512)
        self.fc2 = nn.Linear(512, 256)
        self.fc3 = nn.Linear(256, 1)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.2)
        
    def forward(self, x):
        x = self.dropout(self.relu(self.fc1(x)))
        x = self.dropout(self.relu(self.fc2(x)))
        x = self.fc3(x)
        return x
 

The listing shows how the two hidden layers and their activations are chained in the forward method. section 8.2.2

Weights are initialized to small random values so neurons in the same layer can learn different features. If all weights start at the same constant, those neurons receive the same gradients and update in the same way. section 8.2.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains neuron computations, layer dimensions, parameter shapes, and the rationale for random weight initialization.
Examples
Demonstrates a kinase binder network with 2,048 inputs and hidden layers of 512 and 256 neurons.
What you can do after
Apply it via PyTorch by defining hidden linear layers and chaining ReLU and dropout in a forward method.
How it connects
Connects hidden layers to activation functions, logistic regression, computational graphs, backpropagation, and dropout.

Also explained in

Compare how 11 books explain hidden layer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.