liveBook · Manning

Message passing neural network

As explained in Sutskever's List

A message passing neural network is a neural network architecture that represents graph-structured data by iteratively passing and aggregating information between neighboring nodes.

A molecule poses a problem for networks built around grids or sequences: its atoms and bonds form a graph. In 2017, Justin Gilmer and colleagues used message passing neural networks (MPNNs) to learn molecular properties directly from that structure, rather than relying on hand-engineered molecular descriptors. section 7.3 section 7.3.2

Each atom is a node with a hidden state; bonds are edges that can carry information such as distance. During a message-passing round, neighboring atoms exchange learned messages, and each atom updates its state from what it receives. Repeated rounds let a state summarize a wider neighborhood. A readout then pools the atom states to predict a property of the whole molecule, such as its energy or dipole moment. section 7.3.2

This is also a communication problem. Information from a distant atom may need many rounds to arrive, especially in a sparse graph. The researchers explored virtual edges between otherwise unconnected atoms and master nodes to help information travel; virtual edges remain distinguishable from physical bonds. section 7.3.2

Predicting properties on QM9

QM9 contains about 134,000 small molecules with computed target properties. In tests of MPNN variants, bond type plus spatial distance and explicit hydrogen nodes mattered substantially. The strongest single model combined an edge-network message function with a set2set readout; averaging five models improved results further. The models set new best results on all 13 targets and reached chemical accuracy on 11, while producing predictions in milliseconds rather than the hours costly calculations can require. section 7.3.3 section 7.3.5

Table 7.3 QM9 benchmark comparison across 13 molecular targets. Rows list the predicted properties: dipole moment (mu), polarizability (alpha), HOMO, LUMO, band gap, electronic spatial extent (R2), zero-point vibrational energy (ZPVE), atomization-energy targets (U0), U, H, G, heat capacity (Cv), and highest vibrational frequency (Omega). Columns compare hand-engineered descriptor methods, prior graph-based baselines, and the paper’s best message-passing models. Each cell reports error as a ratio to the target’s chemical-accuracy threshold, so values below 1.0 indicate chemical accuracy. The edge-network (“enn”) + set2set (“s2s”) model (“enn-s2s”) with explicit hydrogens is the best single model across all targets, and the five-model ensemble (“enn -s2s-ens5”) further improves performance.
Target
BAML
BOB
CM
ECFP4
HDAD
GC
GG-NN
DTNN
enn-s2s
enn-s2s-ens5
mu
4.34
4.23
4.49
4.82
3.34
0.70
1.22
-
0.30
0.20
alpha
3.01
2.98
4.33
34.54
1.75
2.27
1.55
-
0.92
0.68
HOMO
2.20
2.20
3.09
2.89
1.54
1.18
1.17
-
0.99
0.74
LUMO
2.76
2.74
4.26
3.10
1.96
1.10
1.08
-
0.87
0.65
gap
3.28
3.41
5.32
3.86
2.49
1.78
1.70
-
1.60
1.23
R2
3.25
0.80
2.83
90.68
1.35
4.73
3.99
-
0.15
0.14
ZPVE
3.31
3.40
4.80
241.58
1.91
9.75
2.52
-
1.27
1.10
U0
1.21
1.43
2.98
85.01
0.58
3.02
0.83
-
0.45
0.33
U
1.22
1.44
2.99
85.59
0.59
3.16
0.86
-
0.45
0.34
H
1.22
1.44
2.99
86.21
0.59
3.19
0.81
-
0.39
0.30
G
1.20
1.42
2.97
78.36
0.59
2.95
0.78
0.84
0.44
0.34
Cv
1.64
1.83
2.36
30.29
0.88
1.45
1.19
-
0.80
0.62
Omega
0.27
0.35
1.32
1.47
0.34
0.32
0.53
-
0.19
0.15
Average
2.17
2.08
3.37
53.97
1.35
2.59
1.36
-
0.68
0.52

Compare errors across 13 QM9 targets against descriptor pipelines, earlier graph models, the best single MPNN and its ensemble; ratios below 1 meet the target’s chemical-accuracy threshold. section 7.3.3

Compute still matters. A towers architecture split a wide state into narrower message passes with occasional mixing; eight towers gave roughly a twofold speedup for a 200-dimensional state on QM9 without meaningful accuracy loss. Later models, including SchNet and DimeNet, added more geometric and physical context. The practical lesson is to compute over entities and links when the data is relational—but to judge the design by measured accuracy and speed, not by the appeal of graph-based reasoning alone. section 7.3.4 section 7.3.5 section 7.6

How Sutskever's List covers it

How deep it goes
Explains node updates, repeated message passing, graph readout, long-range communication, and selected architectural and training trade-offs without a formal derivation.
Examples
Compares QM9 predictions across 13 molecular targets, descriptor pipelines, earlier graph models, a single edge-network model, and its ensemble.
What it assumes
Introduces molecular graphs as atoms and bonds by contrasting them with the grids and sequences used by CNNs and RNNs.
What you can do after
Evaluate/understand it by weighing molecular-prediction accuracy and inference speed against hand-engineered descriptors and costly calculations.
How it connects
Connects graph neural networks to molecular descriptors, pooling, virtual edges, density functional theory, and physics-informed models such as SchNet and DimeNet.
Cover of Sutskever's List
liveBookmanning.com
Read Sutskever's List on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Sutskever's List. The section links point to the parts of the book it draws on.