A graph attention network (GAT) is a graph neural network that learns how strongly each neighboring node should influence a target node. Instead of treating normalized neighbors equally, it assigns learnable attention weights and aggregates a weighted neighborhood sum. This is useful when relationships differ in relevance, graphs contain noisy connections, or node relationships are nonlinear. section 11.6.2
A practical GAT pipeline connects attention-based encoding to a downstream prediction task:
- Represent domain objects and their interactions as a graph, such as transactions connected by Bitcoin flows or users connected to movies through ratings. section 12.0
- Encode node features by aggregating neighboring representations with learned attention coefficients rather than uniform neighborhood contributions. section 11.6.2
- Pass the encoder output to a task-specific decoder; node classification uses log_softmax with CrossEntropyLoss, while link prediction evaluates predicted interactions. section 12.1.4 section 12.0
PyTorch Geometric encoder
A two-layer GAT implementation uses GATConv layers. The first layer produces hidden representations with eight attention heads and concatenates them; the second uses one head and disables concatenation to produce the output dimension. Self-loops can be enabled or disabled through add_self_loops. section 12.1.4
Listing 12.13 GAT model implemented by extending the base class
from torch_geometric.nn import GATConv
class GAT(BaseGraphModel):
def __init__(self, input_dim,
hidden_dim,
out_dim,
num_heads=8,
add_self_loops=True): #1
concat_hidden_dim = hidden_dim * num_heads #2
conv1 = GATConv( #3
in_channels=input_dim,
out_channels=hidden_dim,
heads=num_heads,
add_self_loops=add_self_loops
)
conv2 = GATConv( #4
in_channels=concat_hidden_dim,
out_channels=out_dim,
heads=1, #5
add_self_loops=add_self_loops,
concat=False #6
)
super(BaseGraphModel, self).__init__() #7
self.conv1 = conv1 #8
self.conv2 = conv2 #9
Inspect the complete GAT model definition, including multi-head dimensions, self-loop configuration, and the two convolutional layers. section 12.1.4
Attention adds learnable coefficients and therefore increases parameter count and training cost. In the reported comparisons, GAT was the least efficient encoder for both node classification and link prediction; precision, recall, F1-score, and confusion matrices support evaluation across classes and recommendations. section 12.1.5 section 12.2.5
