liveBook · Manning

Graph attention network

As explained in Fight Fraud with Machine Learning

A graph attention network is a graph neural network that updates node representations by weighting information from neighboring nodes with an attention mechanism.

A graph attention network (GAT) is a graph neural network for classifying nodes by combining each node’s features with information from its neighbors. Unlike a graph convolutional network (GCN), which averages neighbor features equally, a GAT learns how much weight to give each neighbor. This can distinguish nodes with the same neighbors but different connection patterns. section 9.4.1 section 9.5

Figure 9.22 Comparison of a GAT model layer with a GCN model layer, showing how the GAT model uses the attention mechanism to derive different weights/importances for different neighbors of a given node.

The layer comparison shows the key shift: replace equal neighbor averaging with learned, neighbor-specific importance weights before aggregation. section 9.4.1

A GAT layer uses a shared trainable attention vector to calculate an attention coefficient for every node–neighbor pair. Those coefficients become custom averaging weights, so the model can emphasize neighbors and feature dimensions that are more useful for deciding a node’s class. section 9.4.1

A practical illicit-transaction workflow is:

  • Start with graph node features and an edge index representing transaction neighborhoods; use an untrained model’s scattered predictions as a baseline. section 9.4 section 9.4.3
  • Replace GCNConv layers with PyG GATConv layers, then select hidden channels and the number of attention heads. section 9.4.2
  • Train with the existing graph-model loop, monitor loss, validation accuracy, and F1 score, then evaluate on an unseen test set. section 9.4.4
  • Visualize test predictions to check whether illicit and licit transactions separate in two dimensions. section 9.4.5

Multi-head GAT for Bitcoin transactions

A two-layer PyG model accepts 165 input features per transaction. Its first GATConv layer uses 4 heads that each produce 8 features; concatenation produces 32 features for a second, single-head layer that outputs licit and illicit classes. Multiple heads perform independent neighborhood aggregations, giving the next layer several learned views of the graph. section 9.4.2

On the Elliptic Bitcoin dataset, attention-weighted aggregation improved reported test performance over the earlier MLP and GCN baselines: the GAT reached 95.74% accuracy and a 95.46% F1 score in one evaluation. A trained prediction plot showed clearer separation of illicit from licit transactions. These results are dataset-specific, so retain held-out evaluation and learning-curve checks when adapting the model. section 9.4.4 section 9.4.5

GATs aggregate all neighbors, which can become difficult for nodes with very large neighborhoods. GraphSAGE is a related graph neural network approach that instead samples a subset of neighbors during training for large graphs. section 9.4.5 section 9.5

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains attention coefficients and multi-head aggregation, then implements, trains, evaluates, and visualizes a PyG model.
Examples
Uses the Elliptic Bitcoin transaction graph, a 4-head PyG architecture, learning curves, F1 scores, and t-SNE prediction plots.
What it assumes
Builds from GCN neighbor aggregation and introduces attention-weighted aggregation as its replacement.
What you can do after
Apply it via PyG to classify graph nodes, evaluate accuracy and F1, and inspect prediction separation.
How it connects
Connects GATs to graph neural networks, GCNs, attention mechanisms, MLP baselines, and GraphSAGE.

Also explained in

Compare how 4 books explain graph attention network →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.