Model architecture is the arrangement of layers, nodes, and connections that determines how a neural network transforms inputs into outputs. A network has an input layer, hidden layers, and an output layer. section 10.8
A small architecture
Imagine a network with an input layer of size 4, hidden layers of sizes 5 and 3, and an output layer of size 1. Its depth is 3 because the input layer is excluded from the count. In a fully connected architecture, every node connects to every non-bias node in the next layer, with no connections between nonconsecutive layers. section 10.8
Architecture describes several choices:
- Layer sizes specify how many non-bias nodes each layer contains. section 10.8
- Depth specifies the number of layers after the input layer. section 10.8
- Connections determine whether the network is fully connected or uses a different arrangement. section 10.21 section 10.8
- Activation functions help determine the behavior of nodes, while layer count and node count are tunable hyperparameters. section 10.16
Figure 10.14 The general architecture of a neural network. The nodes are divided into layers, where the leftmost layer is the input layer, the rightmost layer is the output layer, and all the layers in between are hidden layers. All the nodes in a layer are connected to all the (non-bias) nodes in the next layer.
Use the layer diagram to trace inputs through hidden layers to outputs and see the fully connected pattern. section 10.8
Adding hidden layers can produce increasingly complex classifier boundaries. Choosing the right architecture is difficult; a larger network may be paired with regularization or dropout to reduce overfitting. section 10.8 section 10.13





















