A linear layer turns input features into output features using learned weights and, by default, a bias. In PyTorch, `nn.Linear(1, 1)` needs one weight and one bias to turn a single measurement into a single prediction. Pass data to the module by calling it, such as `model(x)`, rather than calling `model.forward(x)` directly. section 6.2.2 section 6.2.1
The constructor takes the number of input features, the number of output features, and an optional bias setting. Arrange multiple samples as a tensor of shape `batch size × input features`; the result has shape `batch size × output features`. For example, `nn.Linear(1, 1)` can process ten measurements together from a tensor shaped `10 × 1`. section 6.2.2
Fit one measurement to another
Create `model = nn.Linear(1, 1)`, give `model.parameters()` to `optim.SGD`, and compare `model(inputs)` with targets using `nn.MSELoss()`. During training, clear gradients, call `loss.backward()`, and take an optimizer step. The layer’s weight and bias are the fitted parameters. section 6.2.2
Linear layers also form fully connected classifiers. A `32 × 32` RGB image can be flattened into 3,072 features, then passed through `nn.Linear(3072, 512)`, an activation, and `nn.Linear(512, 2)` for two classes. The activation matters: without a nonlinearity between the layers, the network remains a linear model. section 7.2.2
Flattening an image this way discards its spatial arrangement. A fully connected layer can combine any pixels, but it must learn shifted versions of a pattern separately. A convolutional network instead processes spatial features first, then flattens them for final linear layers. At that handoff, feature counts must match: an `8 × 8 × 8` result becomes 512 input features. section 7.2.6 section 8.2.4
# In[27]:
class Net(nn.Module):
def __init__(self):
super().__init__()
self.conv1 = nn.Conv2d(3, 16, kernel_size=3, padding=1)
self.act1 = nn.Tanh()
self.pool1 = nn.MaxPool2d(2)
self.conv2 = nn.Conv2d(16, 8, kernel_size=3, padding=1)
self.act2 = nn.Tanh()
self.pool2 = nn.MaxPool2d(2)
self.fc1 = nn.Linear(8 * 8 * 8, 32)
self.act3 = nn.Tanh()
self.fc2 = nn.Linear(32, 2)
def forward(self, x):
out = self.pool1(self.act1(self.conv1(x)))
out = self.pool2(self.act2(self.conv2(out)))
out = out.view(-1, 8 * 8 * 8) #1
out = self.act3(self.fc1(out))
out = self.fc2(out)
return out
The network keeps the batch dimension while flattening pooled features, then passes 512 features through two linear layers to produce two outputs. section 8.3.1
The same feature-mapping role appears beyond images: flattened character embeddings feed linear layers for next-character prediction, and linear transformations produce value vectors in self-attention. section 9.4 section 9.5.1
