Depth is the number of layers in a neural network. In a GNN, each layer is a message-passing step that lets every node aggregate information from one additional hop of neighbors, expanding its receptive field. section 3.3.2
Choose depth by balancing neighborhood coverage against representation quality and computational cost:
- Too few layers restrict each node to a small neighborhood and may produce weak representations. section 3.3.2
- More layers reach farther nodes but can cause over-smoothing, making node features too similar to distinguish. section 3.3.2
- Two- or three-layer GNNs often balance neighborhood information with over-smoothing and complexity; very deep models can also overfit. section 3.3.2
Comparing shallow and deep models
Training curves compare GNNs with 2, 10, and 50 layers. The 2-layer model has the strongest overall profile; the 10-layer model improves training loss but shows higher validation loss, while the 50-layer model degrades on both measures. section 3.3.2
A practical GraphSAGE implementation exposes depth through a configurable number of convolution layers. A refined baseline keeps two layers, combines aggregation experiments with 50% dropout, and outperforms the default mean-aggregation model in reported tests. section 3.3.3
Listing 3.7 GraphSAGE class
class GraphSAGEWithCustomDropout(torch.nn.Module):
def __init__(self, in_channels, \
hidden_channels, out_channels, num_layers, \
dropout_rate=0.5, aggr='mean'): #1
super(GraphSAGEWithCustomDropout, self).__init__()
self.layers = torch.nn.ModuleList\
([SAGEConv(in_channels, hidden_channels, aggr=aggr)])
for _ in range(1, num_layers-1): #2
self.layers.append(SAGEConv\
(hidden_channels, hidden_channels, aggr=aggr))
self.layers.append(SAGEConv\
(hidden_channels, out_channels, aggr=aggr))
self.dropout_rate = dropout_rate
def forward(self, x, edge_index):
for layer in self.layers[:-1]:
x = F.relu(layer(x, edge_index))
x = F.dropout(x, p=self.dropout_rate, training=self.training)
x = self.layers[-1](x, edge_index)
return F.log_softmax(x, dim=1)
Inspect the configurable GraphSAGE class and model definitions to see how layer depth is selected alongside dropout and aggregation. section 3.3.3
