Linear layer
A linear layer is a neural-network layer that maps input features to outputs by a learned affine transformation, typically connecting each output to every input feature.
Inside Deep Learning: Math, Algorithms, Models
A PyTorch module that applies an affine linear transformation to its input.
- How it is used
- Deploys affine layers for regression, classification, hidden representations, and feed-forward network construction.
- How deep it goes
- Explains weights, bias, dimensions, module composition, and weight reuse, but defers lower-level gradient implementation.
- Examples
- Uses dimensioned layer pairs, an MNIST classifier, reused hidden layers, and a transpose-weight PCA autoencoder.
- What you can do after
- Build it from scratch by choosing input and output dimensions, composing layers, adding nonlinearities, and reusing layer objects.
Read how Inside Deep Learning: Math, Algorithms, Models explains linear layer →
Deep Learning with Python, Third Edition
- How it is used
- Uses Dense layers to process vector data and provide final classification or regression outputs.
- How deep it goes
- Works through weight and bias shapes and an explicit implementation, but leaves basic tensor operations and backpropagation unimplemented.
- Examples
- Demonstrates a 784-to-32 feature mapping, a sentiment-classification stack, and an activation-free scalar-regression output.
- What you can do after
- Build it from scratch as a small Keras-backed class, or apply it via Keras `Dense` with units and activation chosen for the task.
Read how Deep Learning with Python, Third Edition explains linear layer →
Deep Learning with PyTorch, Second Edition
A PyTorch linear layer module that applies an affine transformation to map features to outputs.
- How it is used
- Maps features to predictions in scalar regression, image classifiers, and character models.
- How deep it goes
- Works through constructor arguments, parameters, batched shapes, training, and the flattening needed after convolutional layers.
- Examples
- Uses temperature fitting, a flattened bird-versus-airplane classifier, and a convolutional classifier with a two-output head.
- What you can do after
- Apply it via PyTorch nn.Linear to fit scalar data or build classifier heads with matching feature dimensions.
Read how Deep Learning with PyTorch, Second Edition explains linear layer →
Deep Learning with R, Third Edition
A Dense layer is a neural-network layer in which every output unit is connected to every input unit through learned weights and biases.
- How it is used
- Builds feature representations and prediction outputs in classification, regression, and forecasting models.
- How deep it goes
- Explains weight shapes, matrix multiplication, bias, activations, and a custom layer implementation without a formal derivation.
- Examples
- Uses handwritten-digit classification, sentiment classification, scalar regression, and a flattened timeseries forecasting baseline.
- What you can do after
- Build it from scratch with weights and a forward pass, or apply it via R/Keras `layer_dense()` in a model.
Read how Deep Learning with R, Third Edition explains linear layer →
Math and Architectures of Deep Learning
- How it is used
- Connects network stages and produces hidden features, regression outputs, or classification scores.
- How deep it goes
- Defines all-to-all weighted connectivity and relates it to matrix notation, activations, and manifold transformations.
- Examples
- Uses demand prediction, height-and-weight classification, LeNet digit classification, and Faster R-CNN prediction heads.
- What you can do after
- Apply it via PyTorch torch.nn.Linear when assembling and training regression or classification networks.
Read how Math and Architectures of Deep Learning explains linear layer →
Grokking Machine Learning
A neural-network architecture in which every node in a layer connects to every non-bias node in the next layer, with no connections between nonconsecutive layers.
- How it is used
- Describes how nodes connect in a layered neural-network architecture.
- How deep it goes
- Defines layer connectivity, layer sizes, and depth, while pointing to other architectures without detailing them.
- Examples
- Uses a network with layer sizes 4, 5, 3, and 1 to illustrate depth.
- What you can do after
- Understand how to identify layer connections and count network depth.
Natural Language Processing in Action, Second Edition
A fully connected linear layer is a neural-network layer that computes an affine transformation from every input activation to its output units.
- How it is used
- Maps compressed neural representations to prediction-sized outputs in CNNs, RNNs, LSTMs, and transformer blocks.
- How deep it goes
- Explains dimensional sizing and architectural placement, but stops short of deriving the affine computation or optimization internals.
- Examples
- Uses pooled CNN text features, surname-nationality RNN predictions, LSTM hidden states, and transformer feedforward layers.
- What you can do after
- Build it from scratch with Linear layers, size inputs from preceding tensor shapes, and connect representations to classification or regression targets.
Read how Natural Language Processing in Action, Second Edition explains linear layer →
Learn Generative AI with PyTorch
A PyTorch layer that applies a learned linear transformation to its input.
- How it is used
- Uses learned feature transformations inside classifiers, sequence models, autoencoders, generators, and discriminators.
- How deep it goes
- Explains input and output dimensions, stacking, activations, and image flattening, but does not derive the transformation mathematically.
- Examples
- Demonstrates clothing classification, two-dimensional discrimination, text generation, autoencoding handwritten digits, and image generation.
- What you can do after
- Build it from scratch with nn.Sequential by matching tensor dimensions and combining linear layers with task-appropriate activations.
Read how Learn Generative AI with PyTorch explains linear layer →
Sutskever's List
- How it is used
- Uses dense layers after convolution to combine image features into ImageNet class predictions.
- How deep it goes
- Explains dense connectivity, flattening, parameter concentration, and output scoring, but gives no weight-level calculations.
- Examples
- Uses AlexNet’s three fully connected layers, 4,096-unit hidden layers, 1,000-way output, and 50% dropout in FC6 and FC7.
- What you can do after
- Understand how dense layers aggregate image features into classification scores and probabilities.
Evolutionary Deep Learning: Genetic algorithms and neural networks
A dense layer is a neural-network layer in which each unit receives inputs from all units in the preceding layer.
- How it is used
- Connects flattened CNN features to class outputs and represents dense-layer choices in evolutionary architecture encodings.
- How deep it goes
- Shows Keras construction and neuron-count encoding, but does not derive the layer's mathematical internals.
- Examples
- Uses a Fashion-MNIST CNN head with 128 hidden neurons and 10 outputs, plus a gene sequence encoding a 25-neuron dense layer.
- What you can do after
- Build it from scratch via Keras and encode its neuron count for generated CNN architectures.
Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains linear layer →
Transformers in Action
nn.Linear is a PyTorch module that applies an affine linear transformation to its input.
- How it is used
- Transforms fixed-length vectors into representations suitable for feed-forward networks, transformer embeddings, and prediction heads.
- How deep it goes
- Explains full connectivity, dimensional expansion and contraction, nonlinear placement, and independent reuse across sequence positions.
- Examples
- Uses a position-wise FFN, a vision-transformer patch projection, and a masked-token prediction head.
- What you can do after
- Build it from scratch as a PyTorch module inside an FFN or as a projection from patches or hidden states.
Build a Text-to-Image Generator (from Scratch)
A fully connected layer is a neural-network layer in which every output unit receives input from every unit in the preceding layer.
- How it is used
- Uses learned projections, feed-forward transformations, and classifier heads within attention, transformer, and image-classification architectures.
- How deep it goes
- Explains placements, dimensions, and surrounding operations, but does not derive parameter gradients or matrix multiplication internals.
- Examples
- Demonstrates attention projections, a 48-to-192-to-48 transformer MLP, ten-class ViT logits, and 1,000-class ResNet50 logits.
- What you can do after
- Builds linear transformations in PyTorch using nn.Linear and places them within attention, MLP, or classification components.
Read how Build a Text-to-Image Generator (from Scratch) explains linear layer →
Grokking AI Algorithms, Second Edition
A linear layer is a neural-network layer that computes an affine or linear transformation of its input using learned weights and usually a bias.
- How it is used
- Transforms token representations inside feed-forward blocks and maps final outputs to vocabulary scores for next-token prediction.
- How deep it goes
- Traces matrix multiplication, bias addition, ReLU, logits, softmax, and argmax, but avoids deeper parameter-learning internals.
- Examples
- Uses NumPy projection code, sample weight tables, token vectors, a logit calculation, and vocabulary probability tables.
- What you can do after
- Build a simple NumPy projection and interpret its logits as scores for selecting a next token.
Read how Grokking AI Algorithms, Second Edition explains linear layer →
Related concepts
- affine transformimplements · An affine transformation is a mapping that combines a linear transformation with an additive translation or offset.
- feedforward networkpart of · A feedforward network is a neural network or subnetwork in which information flows from inputs toward outputs through successive computations without recurrent connections or cycles.
- logitproduces · A logit is a real-valued quantity representing an outcome’s log-odds or, in machine learning, an unnormalized model score used to derive probabilities.
- matrix-matrix multiplicationuses · Matrix multiplication combines compatible matrices to form a result whose entries are sums of products between rows of the first matrix and columns of the second.
- softmaxuses · Softmax is a function that exponentiates a vector of scores and normalizes the results into nonnegative values that sum to one.
- neural network layerkind of · A neural network layer is a computational stage or component that transforms input representations into outputs within a neural network.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.