Mean pooling forms one fixed-size representation by averaging the vectors produced for sequence or graph positions. In molecular graphs, it converts atom-level features into one molecule-level vector, regardless of atom count. In protein classifiers, it converts token-level representations into one sequence vector for classification. section 11.5.2 section 12.3.4
The operation is simple, but its implementation must preserve which positions contain real data:
- Graph batches use a batch vector so the mean is computed separately for each molecule rather than across every node in the mini-batch. section 11.5.2
- Sequence batches use an attention mask to exclude padding tokens, preventing meaningless values from diluting the average. section 12.3.4
- Mean pooling averages all positions equally; unlike sum pooling it is automatically size-normalized, and unlike max pooling it is less sensitive to outliers. section 11.5.2 section 12.3.4
Masked sequence pooling
For token embeddings shaped by batch and sequence position, expand the attention mask, multiply it into the embeddings, sum across positions, and divide by the masked token count. Clamping the count prevents division by zero. The resulting fixed-size vector can feed a classifier for antibody sequences. section 12.3.4
x = global_mean_pool(x, mol_batch)
Use the compact PyTorch Geometric operation to convert node features into one representation per molecular graph. section 11.5.2
Listing 12.2 Average Token Embeddings
def mean_pooling(self, token_embeddings, attention_mask):
expanded_mask = attention_mask.unsqueeze(-1).expand(
token_embeddings.shape
).float() #A
sum_embeddings = torch.sum(
token_embeddings * expanded_mask, dim=1
) #B
num_tokens = torch.clamp(
expanded_mask.sum(1), min=1e-9
) #C
return sum_embeddings / num_tokens #DInspect the complete masked PyTorch implementation for averaging variable-length sequence representations while ignoring padding. section 12.3.4

