Neural relational inference

As explained in Graph Neural Networks in Action

Neural relational inference is a neural model that infers latent relational structure and dynamics from observed system data.

Neural relational inference (NRI) predicts how interacting entities move while inferring which entities interact. It is useful when observations arrive over time but the interaction network is partly or entirely unknown. Unlike a temporal-only RNN or a graph model that collapses time into one graph, NRI combines relational and temporal learning. section 6.4 section 6.4.3

Think of each observed frame as node data for a changing graph:

• Encode the node data into a probability distribution over latent edge types. In a body-pose example, the two types are connected and not connected. section 6.5.3
• Sample edges from that distribution and use a decoder with recurrent memory to reconstruct subsequent graph states and node trajectories. section 6.4 section 6.5.3
• Train against the next observed states using a reconstruction term and a KL-divergence term. For longer forecasts, encode the observed frames, then run the decoder forward without teacher forcing. section 6.4

Figure 6.11 Schematic for NRI (Source: Kipf et al. [2]). The model consists of an encoder and decoder layer and several message-passing steps. However, here the messages are passed in the encoder from node to edge, back from edge to node, and then back from node to edge again. For the decoder, messages are passed from node to edge and then from edge to node. The final step takes the latent representation and is used to predict the next step in the temporal evolution of the body.

The schematic traces message passing between nodes and edges in the encoder and decoder, then shows how the latent representation produces a future step. section 6.4

Predicting a walking pose

Treat joint sensors as nodes and body connections as edges. Given earlier poses, predict later sensor positions. On the reported test, NRI reached an MSE of 0.00008 for the next timestep and 94% accuracy at 40 timesteps, versus 65% for an LSTM and 55% for a GAT. Accuracy still decreased as the forecast extended. section 6.6 section 6.4.3

The same joint inference matters beyond poses: moving particles or players may interact only under conditions that are not supplied as graph edges. The two-edge-type pose setup is a starting point, not a requirement; more interaction types are possible. section 6.4.3 section 6.5.3

How Graph Neural Networks in Action covers it

How deep it goes

Builds an encoder–decoder prediction pipeline with edge sampling, recurrent decoding, and loss terms, without a rigorous mathematical derivation.

Examples

Uses joint-sensor pose forecasting, training and evaluation listings, and comparisons with LSTM and GAT predictions.

What it assumes

Assumes familiarity with graph embeddings, autoencoders, and recurrent models, while walking through NRI’s components using node and edge data.

What you can do after

Build it from scratch to predict future graph states and infer interaction types from observed node trajectories.

How it connects

Combines graph autoencoders, message passing, recurrent neural networks, latent edge types, and spatiotemporal GNNs.

liveBookmanning.com

Read Graph Neural Networks in Action on liveBook Get the book at manning.com

Related concepts

• recurrent neural networkuses · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
• autoencoderuses · An autoencoder is a neural network that encodes input data into an internal representation and decodes it to reconstruct the input or an approximation.
• Kullback–Leibler divergenceuses · Kullback–Leibler divergence is a nonnegative, directed measure of the information lost when one probability distribution approximates another, computed as an expected log ratio under the first distribution.
• decoderhas part · A decoder is a component that maps encoded data or representations to decoded values, reconstructed data, or generated output.
• encoderhas part · An encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
• variational autoencoderuses · A variational autoencoder is a generative neural network that learns a probabilistic latent representation of data and decodes samples from it to reconstruct or generate data.

This explainer was generated by Manning's knowledge graph from the text of Graph Neural Networks in Action. The section links point to the parts of the book it draws on.