liveBook · Manning

JumpingKnowledge

As explained in Graph Neural Networks in Action

A PyTorch Geometric layer that combines outputs from multiple neural-network layers using modes such as concatenation, max pooling, or learned attention.

JumpingKnowledge combines representations produced by multiple graph-convolution layers, letting a model retain shallow and deeper node features instead of using only the final layer. section 3.2.2

Choose the aggregation mode according to the information and dimensionality trade-off:

  • cat concatenates every layer output, preserving information while increasing feature dimensionality. section 3.2.2
  • max takes the largest value for each feature across layers, emphasizing strong signals and reducing the effect of less informative outputs. section 3.2.2
  • lstm uses a bidirectional LSTM to learn layer-attention scores and combine outputs dynamically. section 3.2.2

Two-layer GCN pipeline

Create JumpingKnowledge(mode='cat'), collect the activated output of the first GCNConv and the output of the second, then pass the list to self.jk. In the reported comparison, JK improved log loss but had a lower F1 score than the baseline GCN. section 3.2.2

Listing 3.6 GCN class with JumpingKnowledge layer
class CustomGCN(torch.nn.Module):
   def __init__(self, in_channels, hidden_channels, out_channels):
       super(CustomGCN, self).__init__()
       self.conv1 = GCNConv(in_channels, hidden_channels)
       self.conv2 = GCNConv(hidden_channels, out_channels)

       self.jk = JumpingKnowledge(mode='cat')  #1

   def forward(self, x, edge_index):
       layer_outputs = []  #2

       x1 = self.conv1(x, edge_index)
       x1 = F.relu(x1)
       layer_outputs.append(x1)  #3

       x2 = self.conv2(x1, edge_index)
       layer_outputs.append(x2) 


       x = self.jk(layer_outputs)  #4

       return x

Browse the complete CustomGCN implementation showing initialization, layer-output collection, and JumpingKnowledge aggregation. section 3.2.2

How Graph Neural Networks in Action covers it

How deep it goes
Explains three modes and a two-layer implementation, but does not derive their underlying optimization or attention mechanics.
Examples
Uses a CustomGCN implementation, layer-output lists, and baseline-versus-JK F1 and log-loss results.
What it assumes
Assumes graph-convolution layers, node features, edge_index, and forward passes; demonstrates output collection from multiple layers.
What you can do after
Apply it via PyTorch Geometric by collecting intermediate layer outputs and selecting an aggregation mode.
How it connects
Connects graph convolutional networks, GCNConv, GraphSAGE, neighborhood aggregation, and node classification.
Cover of Graph Neural Networks in Action
liveBookmanning.com
Read Graph Neural Networks in Action on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Graph Neural Networks in Action. The section links point to the parts of the book it draws on.