nn.Linear applies an affine transformation: it combines input features with learned weights and adds a learned bias. Calling nn.Linear(d, C) creates a module with d inputs and C outputs, making it useful for regression, classification, and hidden layers in larger networks. section 2.1.5 section 2.7
For an input x, the operation can be written y = xW + b. W contains one set of feature connections for each output, while b shifts outputs independently of x. A layer such as nn.Linear(3, 4) maps three input features to four outputs; nn.Linear(4, 1) then maps those four values to one prediction. section 2.1.5 section 2.2.1
Use nn.Linear to:
- Implement linear or logistic regression with a single layer. section 2.7
- Build fully connected networks by stacking linear layers and placing nonlinearities between them. section 2.7
- Represent feed-forward connections in which every output receives information from every feature in the preceding layer. section 2.2.1
- Reuse one layer object when tied weights are required across multiple network positions. section 4.1.1
A compact fully connected classifier
D = 28 * 28 n = 256 classes = 10 model = nn.Sequential( nn.Flatten(), nn.Linear(D, n), nn.Tanh(), nn.Linear(n, n), nn.Tanh(), nn.Linear(n, classes), ) Here, Flatten converts each image into D features, the hidden layers transform those features, and the final nn.Linear produces one output per class. The same kind of layer can also be reused for tied weights or paired with a custom transpose layer in a PCA-style autoencoder. section 4.1.1 section 7.1.2
Figure 2.4 A simple feed-forward fully connected network with one input layer of d = 3 inputs, a hidden layers with n = 4 neurons, and one output in the output layer. Connections feed directly into the next layer only, and each node in one layer is connected to every neuron in the preceding layer.
Inspect the network diagram to connect nn.Linear dimensions with fully connected edges between successive layers. section 2.2.1
