Sampling selects subsets of nodes, edges, or subgraphs for computation instead of keeping an entire large graph in memory. It makes large-graph training practical by trading extra sampling and batching complexity for lower memory use. section 7.3.1 section 7.7.1
Separate sampling from batching, then combine them in a training pipeline:
- Batching breaks a dataset into smaller groups; sampling determines which nodes or edges enter those groups. section 7.7.1
- Sampled subsets may form connected subgraphs, but they do not have to. Loaders then serve the selected data to the training process. section 7.7.1
- Choose samplers experimentally: different methods affect epoch time, convergence time, information retention, and model generalization. section 7.7.1
Approximating a GCN with NeighborSampler
A standard GCN does not inherently sample, but PyG’s NeighborSampler can act as a dataloader that returns subgraphs from a larger graph. GCNConv processes these subgraphs to approximate full-graph convolution, reducing memory requirements while potentially lowering accuracy. section 7.7
Sampling can omit important message-passing neighborhoods and introduce bias into aggregation. PyG provides predefined loaders and samplers, plus base classes for custom implementations; selecting among them depends on graph structure and training goals. section 7.7.1
Figure 1.14 Mental model of the GNN project. We start with raw data, which is transformed into a graph data model that can be stored in a graph database or used in a graph processing system. From the graph processing system (and some graph databases), exploratory data analysis and visualization can be done. Finally, for graph machine learning, data is preprocessed into a form that can be submitted for training.
Place sampling within the GNN workflow: raw data becomes a graph, preprocessing applies sampling and batching, and the resulting data enters training. section 1.5.1
