liveBook · Manning

Xavier/Glorot initialization

As explained in Graph Neural Networks in Action

A neural-network parameter initialization method that scales initial weights according to the sizes of adjacent layers to help maintain activation and gradient variance.

Xavier initialization gives a GNN informative random starting features when the graph has no predefined or useful node attributes. Its distribution is chosen to keep activation variety consistent across layers, helping avoid vanishing or exploding gradients while embeddings learn. section 2.2.1

Apply it to PyTorch Geometric node features as follows:

  • Create a floating-point feature matrix with one row per node and the desired feature width. section 2.2.1
  • Initialize that matrix with nn.init.xavier_uniform_. section 2.2.1
  • Pass the initialized data.x through the GNN; training adjusts these starting values toward the task objective. section 2.2.1
data.x = torch.randn((data.num_nodes, 64), dtype=torch.float)
'nn.init.xavier_uniform_(data.x) '

The listing shows the PyTorch construction of random node features followed by Xavier uniform initialization. section 2.2.1

How Graph Neural Networks in Action covers it

How deep it goes
Explains distribution-level motivation and PyTorch application, without deriving the initialization distribution.
Examples
Uses data.x with one random feature row per node and nn.init.xavier_uniform_ before SimpleGNN inference.
What it assumes
Assumes tensor-based graph data and introduces random node features for graphs lacking informative attributes.
What you can do after
Apply it via PyTorch to initialize node features for learning task-specific graph embeddings.
How it connects
Connects node features with GNNs, node embeddings, PyTorch Geometric, and task-driven training.
Cover of Graph Neural Networks in Action
liveBookmanning.com
Read Graph Neural Networks in Action on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Graph Neural Networks in Action. The section links point to the parts of the book it draws on.