Hidden layer

As explained in Graph Neural Networks in Action

An intermediate layer of a neural network whose activations are not directly exposed as the model's inputs or outputs.

A hidden layer transforms a network’s input into an intermediate representation before the final output. That representation can be useful in its own right: when a network learns to predict which nodes appear together on graph walks, its hidden layer can provide node embeddings. section 2.4.3 section 2.4.4

What the hidden layer does depends on the task:

• In node2vec, a shallow network predicts visited nodes from a starting node; the trained hidden layer becomes a node embedding. section 2.4.3
• In a multilayer perceptron, hidden layers transform an input vector toward a prediction. Their outputs can also serve another task or model; for node embeddings, the last hidden layer is typically used. section 2.4.4
• In a recurrent neural network, a hidden layer is updated at each timestep using the new input and previous hidden state, allowing sequential data to influence later outputs. section 6.5.1

A hidden layer in PyTorch

A basic recurrent model can define an nn.RNN with a chosen hidden size, initialize its hidden state with zeros, and pass selected RNN outputs through nn.Linear to produce predictions. This is a single-layer starting point; RNNs can also be stacked or replaced with LSTM or GRU layers. section 6.5.1

Figure 6.17 Structure for an RNN. Temporal updates as a single self-loop denoted by Whh (left). An unfolded model in time showing the model updates (right). Here, we change our input, output, and hidden layers (x, y, h) to be temporal variables (x_t, y_t, h_t). At our initial step, t, we update our current hidden layer with input data from x_t and the weights from our previous hidden layer h_t–1 and then use this to output y_t. The weights from ht are then passed to h_t+1 along with the new input at x_t+1 to infer y_t+1.

The unfolded RNN diagram makes the hidden layer’s update across timesteps visible. section 6.5.1

How Graph Neural Networks in Action covers it

How deep it goes

Explains intermediate representations and RNN updates with a PyTorch example, but defers backpropagation-through-time computational details to deep-learning packages.

Examples

Illustrates hidden representations with graph-walk node prediction, image classification, and a single-layer PyTorch RNN.

What it assumes

Assumes some deep-learning familiarity while briefly introducing tensor-to-output layers and multilayer perceptrons.

What you can do after

Apply it via PyTorch’s nn.RNN by initializing a hidden state and mapping RNN outputs through nn.Linear.

How it connects

Connects hidden layers to node2vec, node embeddings, multilayer perceptrons, graph neural networks, and recurrent neural networks.

liveBookmanning.com

Read Graph Neural Networks in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA neural-network layer positioned between the input and output layers whose units transform intermediate representations.
• 

  liveBookmanning.com
  Classic Computer Science Problems in Swift: Essential techniques for practicing programmersA hidden layer is a neural-network layer between the input and output layers whose outputs are not the network's final outputs.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA hidden layer is an intermediate computational layer in a neural network whose units receive inputs from preceding layers and pass transformed values to subsequent layers.
• 

  liveBookmanning.com
  Inside AIA hidden layer is an intermediate neural-network layer that transforms representations between the input and output layers.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA hidden layer is an intermediate neural-network layer whose activations are not directly exposed as the model's input or final output.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA network layer between input and output that transforms representations before later computation, here before attention is applied.
• 

  liveBookmanning.com
  Data Without LabelsA hidden layer is an intermediate neural-network layer whose activations are not directly observed as inputs or outputs.

Compare how 8 books explain hidden layer →

Related concepts

• neural network layerkind of · A neural network layer is a computational stage or component that transforms input representations into outputs within a neural network.
• deep learningpart of · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
• language modelpart of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• neural networkpart of · A neural network is a computational model of interconnected, parameterized units that learns transformations from inputs to outputs.
• recurrent neural networkpart of · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
• dropoutused by · Dropout is a neural-network regularization technique that randomly suppresses units, connections, or activations during training to reduce overfitting.

This explainer was generated by Manning's knowledge graph from the text of Graph Neural Networks in Action. The section links point to the parts of the book it draws on.