A feedforward network passes information from one layer to the next, without feeding it back during prediction. Each layer transforms its input; a fully connected, or dense, layer connects every neuron to every neuron in the preceding layer. section 8.2.1
In a Transformer encoder, a position-wise feedforward network (FFN) applies the same small network independently to each sequence position. It typically uses two linear transformations with a nonlinear activation, such as ReLU or GELU, between them. section 12.3.2
For a protein sequence, multi-head self-attention first lets each residue gather context from other positions. The FFN then processes the information at each position. Both sublayers use residual connections and layer normalization, and stacking encoder blocks builds increasingly context-aware representations. section 12.3.2
Figure 12.9 The transformer architecture. While the original transformer includes both an encoder and decoder for translation tasks, protein language modeling primarily uses the encoder-only architecture (left). This stack processes the entire sequence in parallel to generate embeddings for every residue.

The encoder stack shows where the position-wise FFN fits in a Transformer used for protein sequences. section 12.3.2
Predicting docking scores
A feedforward surrogate model can predict docking scores from molecular fingerprints, which encode molecular features. A PyTorch implementation uses three fully connected layers, ReLU activations, and dropout at probability 0.5 to help prevent overfitting. This is a molecular-level network, rather than the position-wise FFN in a Transformer. section 9.2.2
Listing 9.5 Deep Learning Surrogate Model for Binding Affinity Prediction
class DeepDockingModel(nn.Module):
def __init__(self, input_size, hidden_size, output_size=1):
super(DeepDockingModel, self).__init__()
self.layers = nn.Sequential( #A
nn.Linear(input_size, hidden_size),
nn.ReLU(),
nn.Dropout(0.5),
nn.Linear(hidden_size, hidden_size // 2),
nn.ReLU(),
nn.Dropout(0.5),
nn.Linear(hidden_size // 2, output_size)
)
def forward(self, x):
return self.layers(x)
def predict(self, fingerprints, device):
self.eval() #B
with torch.no_grad(): #C
inputs = torch.tensor(
np.atleast_2d(fingerprints), dtype=torch.float32
).to(device) #D
outputs = self(inputs) #E
return outputs.cpu().numpy().flatten()The PyTorch listing assembles the surrogate model with nn.Sequential and passes fingerprints through its layers. section 9.2.2
A plain feedforward network over a flattened SMILES string has no built-in notion of sequence order and handles variable-length molecules awkwardly. This differs from a Transformer encoder, where attention provides sequence context before the position-wise FFN processes it. section 10.4 section 12.3.2



















