liveBook · Manning

Neural network layer

As explained in Grokking Machine Learning, Second Edition

A layer is a group of neural-network nodes operating at the same stage between an input and an output.

A neural network layer groups nodes at the same stage between an input and an output. Layers organize how information moves through the network. section 10.8

Identify a layer by its role and structure:

  • The input layer receives features, the output layer produces the result, and hidden layers sit between them. section 10.8
  • The size of a layer counts its non-bias nodes; bias nodes belong to the architecture but are not counted. section 10.8
  • In a fully connected architecture, every node connects to every non-bias node in the next layer, with no connections between nonconsecutive layers. section 10.8
  • The number of layers excluding the input layer gives the network’s depth. section 10.8

A small layered network

Consider a network with an input layer of size 4, hidden layers of sizes 5 and 3, and an output layer of size 1. Its depth is 3 because the input layer is excluded from the count. As layers accumulate classifier boundaries, later layers can represent more complex boundaries than earlier ones. section 10.8

Figure 10.14 The general architecture of a neural network. The nodes are divided into layers, where the leftmost layer is the input layer, the rightmost layer is the output layer, and all the layers in between are hidden layers. All the nodes in a layer are connected to all the (non-bias) nodes in the next layer.

Use the architecture diagram to trace the input, hidden, and output layers and see how neighboring nodes connect. section 10.8

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Defines layer roles, sizes, depth, bias counting, and fully connected structure, while deferring alternative architectures to later treatment.
Examples
Uses a four-five-three-one architecture, layer-boundary visualizations, and TensorFlow Playground for exploring architectures and hyperparameters.
What it assumes
Assumes basic familiarity with nodes and perceptrons while defining layer roles, architecture, depth, and connectivity.
What you can do after
Evaluate/understand a neural network’s layer roles, size, depth, connectivity, and increasing classifier complexity.
How it connects
Connects layers with nodes, bias nodes, architecture, depth, classifiers, perceptrons, activation functions, and neural networks.

Also explained in

Compare how 12 books explain neural network layer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.