Recurrent neural network

As explained in Graph Neural Networks in Action

A recurrent neural network is a neural network architecture that processes sequences using recurrent state updates.

A recurrent neural network (RNN) predicts from a sequence by carrying information from one timestep to the next. It is useful when the recent past matters: given several frames of body-joint sensor positions, an RNN can predict where those sensors will move. This provides a temporal baseline without using the connections between joints. section 6.2.2 section 6.4

Think of the network as the same update repeated across time:

• Start with an initial hidden state. At timestep t, combine the new input with the preceding hidden state to produce an updated state and output. section 6.5.1
• Carry that state into the next timestep alongside the next input. An unfolded view shows these updates as a chain rather than a single recurrent self-loop. section 6.5.1
• Train by backpropagating through time. For future-position prediction, compare outputs with target positions, then use predictions to continue a forecast. section 6.5.1 section 6.2.2

Figure 6.17 Structure for an RNN. Temporal updates as a single self-loop denoted by Whh (left). An unfolded model in time showing the model updates (right). Here, we change our input, output, and hidden layers (x, y, h) to be temporal variables (x_t, y_t, h_t). At our initial step, t, we update our current hidden layer with input data from x_t and the weights from our previous hidden layer h_t–1 and then use this to output y_t. The weights from ht are then passed to h_t+1 along with the new input at x_t+1 to infer y_t+1.

The self-loop and unfolded views show how input and hidden state contribute to successive outputs. section 6.5.1

Pose prediction with PyTorch

Define an nn.RNN with input size, hidden size, layer count, and batch_first=True. Initialize its hidden state for the batch, run the input sequence through the RNN, and pass selected outputs through a linear layer to predict sensor positions. A training loop computes loss, backpropagates, and updates the model; a testing loop generates future predictions. section 6.5.1 section 6.2.2

A basic RNN has limited short-term memory; LSTMs and gated recurrent units (GRUs) are recurrent alternatives for more demanding sequences. An RNN trained only on sensor positions also misses the relationships between sensors. Neural relational inference addresses both kinds of information by combining graph-derived representations with a recurrent decoder. section 6.2.2 section 6.4 section 6.5.3

How Graph Neural Networks in Action covers it

How deep it goes

Explains hidden-state updates and backpropagation through time, then implements an nn.RNN; leaves difficult gradient computations to deep-learning packages.

Examples

Uses body-joint sensor trajectories, an unfolded recurrent-state diagram, and PyTorch model, training, and forecasting code.

What you can do after

Apply it via PyTorch nn.RNN to forecast sensor positions and recognize when missing relational information limits predictions.

How it connects

Connects recurrent networks to LSTMs, GRUs, spatiotemporal GNNs, and neural relational inference.

liveBookmanning.com

Read Graph Neural Networks in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA recurrent neural network is a neural network architecture that processes sequences while maintaining a state influenced by earlier inputs.
• 

  liveBookmanning.com
  Sutskever's ListA recurrent neural network is a neural network architecture that processes sequences by maintaining a state updated across successive inputs.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA neural network architecture that processes sequences by feeding information from one step to the next.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA recurrent neural network is a neural-network model that processes sequences recurrently by carrying information through an internal state across positions.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA neural network that processes sequential data by reusing parameters across time steps and carrying a hidden state forward.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA neural network architecture for sequential data that processes inputs with loops so information can persist across time steps.
• 

  liveBookmanning.com
  LLMs in ProductionA recurrent neural network is a neural network that processes ordered inputs while maintaining a state that carries information across sequence positions.
• 

  liveBookmanning.com
  Grokking Machine LearningA recurrent neural network is a neural network architecture that processes sequences by retaining and updating state across successive inputs.
• 

  liveBookmanning.com
  Data Without LabelsA recurrent neural network is a neural-network architecture designed to model sequences through recurrent state or connections.

Compare how 10 books explain recurrent neural network →

Related concepts

• LSTMincludes · An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.
• sequenceuses · A sequence is an ordered collection of elements whose positions determine their order of traversal or processing.
• language modelkind of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• hidden stateuses · A hidden state is an internal numerical representation in a neural model that encodes information processed up to a particular position, step, or layer.
• bidirectional rnnused by · A bidirectional recurrent neural network processes a sequence in forward and reverse directions and combines the resulting representations.
• statehas part · State is information or a configuration describing the condition of a system, entity, or computation at a particular time and potentially influencing subsequent behavior or decisions.

This explainer was generated by Manning's knowledge graph from the text of Graph Neural Networks in Action. The section links point to the parts of the book it draws on.