A linear layer applies a learned linear transformation to incoming data. In PyTorch, nn.Linear creates the dense layer that maps input features to output features, making it a basic building block for classifiers, sequence models, autoencoders, generators, and discriminators. section 2.3.2 section 8.4
Choose the first layer’s input size to match the input data. In a clothing classifier, each 28 × 28 image is flattened into 784 values before entering nn.Linear(784, 256). Later layers use the preceding layer’s output size as their input size, such as nn.Linear(256, 128). section 2.3.2
A binary image classifier
A sequential classifier can stack linear layers with ReLU activations and finish with nn.Linear(32, 1), followed by sigmoid. The hidden layers use 256, 128, and 32 neurons; the final value is interpreted as the probability that the image is an ankle boot rather than a t-shirt. The same pattern appears in a discriminator, where nn.Linear(2, 256) accepts two input values and nn.Linear(64, 1) produces one output. section 3.3.1
Linear layers need not be the final stage of a model. An LSTM can feed its outputs into a linear layer whose output size equals the vocabulary size, producing logits for the softmax function. Dense layers are also used to build autoencoder encoders and decoders. section 8.4 section 7.2
For images, compare linear layers with convolutional layers. Flattening makes dense processing possible, but using dense layers alone on high-resolution color images can create enormous parameter counts; convolutional layers reduce connections by examining local input regions. section 4.0 section 2.3.2
