liveBook · Manning

Model architecture

As explained in Grokking Machine Learning, Second Edition

The structural organization of layers, processing units, and connections that defines how a neural network transforms inputs into outputs.

Model architecture is the arrangement of layers, nodes, and connections that determines how a neural network transforms inputs into outputs. A network has an input layer, hidden layers, and an output layer. section 10.8

A small architecture

Imagine a network with an input layer of size 4, hidden layers of sizes 5 and 3, and an output layer of size 1. Its depth is 3 because the input layer is excluded from the count. In a fully connected architecture, every node connects to every non-bias node in the next layer, with no connections between nonconsecutive layers. section 10.8

Architecture describes several choices:

  • Layer sizes specify how many non-bias nodes each layer contains. section 10.8
  • Depth specifies the number of layers after the input layer. section 10.8
  • Connections determine whether the network is fully connected or uses a different arrangement. section 10.21 section 10.8
  • Activation functions help determine the behavior of nodes, while layer count and node count are tunable hyperparameters. section 10.16
Figure 10.14 The general architecture of a neural network. The nodes are divided into layers, where the leftmost layer is the input layer, the rightmost layer is the output layer, and all the layers in between are hidden layers. All the nodes in a layer are connected to all the (non-bias) nodes in the next layer.

Use the layer diagram to trace inputs through hidden layers to outputs and see the fully connected pattern. section 10.8

Adding hidden layers can produce increasingly complex classifier boundaries. Choosing the right architecture is difficult; a larger network may be paired with regularization or dropout to reduce overfitting. section 10.8 section 10.13

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Explains layers, depth, bias nodes, connectivity, and classifier boundaries, but defers detailed alternative architectures.
Examples
Uses a 4-5-3-1 fully connected network, layered classifier boundaries, TensorFlow Playground, and a larger-network analogy.
What it assumes
Assumes basic neural-network components while defining layers, hidden layers, depth, bias nodes, and fully connected structure.
What you can do after
Evaluate network diagrams by identifying layers, depth, node counts, connections, and architectural choices.
How it connects
Connects architecture with neural networks, hyperparameters, activation functions, classifier boundaries, regularization, dropout, and overfitting.

Also explained in

Compare how 22 books explain model architecture →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.