liveBook · Manning

Linear layer

Also called fully connected layer, Dense layer, nn.Linear, densely connected layers, fc

A linear layer is a neural-network layer that applies a learned affine transformation to its input, typically using weights and a bias.

How 17 Manning books explain it

Inside Deep Learning: Math, Algorithms, Models

A PyTorch module that applies an affine linear transformation to its input.

How it is used
Provides the input-to-output transformation for regression models, classification heads, and hidden layers in fully connected networks.
How deep it goes
Defines dimensions and bias, builds multilayer examples, and examines weight reuse and linear-operation equivalence; leaves ordinary module backpropagation to PyTorch.
Examples
Uses three-to-four-to-one network dimensions, an MNIST network with shared hidden-layer weights, and a replacement ResNet-18 output layer.
What you can do after
Apply it via PyTorch `nn.Linear` to build prediction layers, compose networks, reuse weights, and replace a pretrained output layer.

Read how Inside Deep Learning: Math, Algorithms, Models explains linear layer →

Build AI Drug Discovery Pipelines

A linear layer is a neural-network layer that applies an affine transformation to its input using a weight matrix and bias vector.

How it is used
Uses linear layers to transform molecular features and graph embeddings into representations and predictions for drug discovery.
How deep it goes
Explains the affine operation, parameter dimensions, layer connectivity, and forward-pass composition in worked PyTorch architectures.
Examples
Illustrates kinase-binding prediction from molecular bit vectors and drug-target affinity prediction from pooled graph representations.
What you can do after
Apply it via PyTorch nn.Linear to assemble dimension-matched layers for molecular prediction.

Read how Build AI Drug Discovery Pipelines explains linear layer →

Deep Learning with Python, Third Edition

How it is used
Uses Dense layers to process vector data and provide classification or regression outputs.
How deep it goes
Implements weights, bias, and optional activation from scratch while leaving tensor operations and backpropagation to existing tools.
Examples
Uses a two-sample shape example, a sentiment classifier, and a scalar housing-price regression model.
What you can do after
Apply Dense layers via Keras to build classification and regression models with task-appropriate outputs.

Read how Deep Learning with Python, Third Edition explains linear layer →

Deep Learning with PyTorch, Second Edition

A PyTorch linear layer module that applies an affine transformation to map features to outputs.

How it is used
Uses linear layers to map features into predictions and connect stages of image and sequence models.
How deep it goes
Explains construction, trainable parameters, batching, optimization, and shape handling through worked models.
Examples
Fits temperature measurements and classifies flattened RGB images with a 3,072-to-512-to-2 network.
What you can do after
Applies PyTorch nn.Linear to batched inputs and incorporates its parameters into a training loop.

Read how Deep Learning with PyTorch, Second Edition explains linear layer →

Deep Learning with R, Third Edition

A Dense layer is a neural-network layer in which every output unit is connected to every input unit through learned weights and biases.

How it is used
Uses Dense layers to transform vector data and build classification and regression models.
How deep it goes
Explains affine computation, activation functions, weight shapes, and a custom Keras implementation.
Examples
Uses MNIST digit classification, review sentiment classification, scalar regression, and temperature forecasting.
What you can do after
Build a simple Dense layer from scratch and apply Dense layers via Keras in classification and regression models.

Read how Deep Learning with R, Third Edition explains linear layer →

Math and Architectures of Deep Learning

How it is used
Connects network stages and produces hidden features, regression outputs, or classification scores.
How deep it goes
Defines all-to-all weighted connectivity and relates it to matrix notation, activations, and manifold transformations.
Examples
Uses demand prediction, height-and-weight classification, LeNet digit classification, and Faster R-CNN prediction heads.
What you can do after
Apply it via PyTorch torch.nn.Linear when assembling and training regression or classification networks.

Read how Math and Architectures of Deep Learning explains linear layer →

Grokking Machine Learning

A neural-network architecture in which every node in a layer connects to every non-bias node in the next layer, with no connections between nonconsecutive layers.

How it is used
Describes how nodes connect in a layered neural-network architecture.
How deep it goes
Defines layer connectivity, layer sizes, and depth, while pointing to other architectures without detailing them.
Examples
Uses a network with layer sizes 4, 5, 3, and 1 to illustrate depth.
What you can do after
Understand how to identify layer connections and count network depth.

Read how Grokking Machine Learning explains linear layer →

Natural Language Processing in Action, Second Edition

A fully connected linear layer is a neural-network layer that computes an affine transformation from every input activation to its output units.

How it is used
Maps compressed neural representations to prediction-sized outputs in CNNs, RNNs, LSTMs, and transformer blocks.
How deep it goes
Explains dimensional sizing and architectural placement, but stops short of deriving the affine computation or optimization internals.
Examples
Uses pooled CNN text features, surname-nationality RNN predictions, LSTM hidden states, and transformer feedforward layers.
What you can do after
Build it from scratch with Linear layers, size inputs from preceding tensor shapes, and connect representations to classification or regression targets.

Read how Natural Language Processing in Action, Second Edition explains linear layer →

Build a DeepSeek Model (From Scratch)

A linear layer is a neural-network layer that maps input representations to outputs through a learned affine transformation.

How it is used
Uses linear layers to remap transformer representations for attention, feed-forward computation, and token prediction.
How deep it goes
Explains projection dimensions and architectural roles, including FFN expansion and contraction, without detailing how the weights are trained.
Examples
Shows a bias-free PyTorch vocabulary head and a 2d-to-d projection of concatenated inputs in multi-token prediction.
What you can do after
Evaluate/understand how linear projections change or preserve dimensions and combine representations across transformer components.

Read how Build a DeepSeek Model (From Scratch) explains linear layer →

Learn Generative AI with PyTorch

A PyTorch layer that applies a learned linear transformation to its input.

How it is used
Uses learned feature transformations inside classifiers, sequence models, autoencoders, generators, and discriminators.
How deep it goes
Explains input and output dimensions, stacking, activations, and image flattening, but does not derive the transformation mathematically.
Examples
Demonstrates clothing classification, two-dimensional discrimination, text generation, autoencoding handwritten digits, and image generation.
What you can do after
Build it from scratch with nn.Sequential by matching tensor dimensions and combining linear layers with task-appropriate activations.

Read how Learn Generative AI with PyTorch explains linear layer →

Sutskever's List

How it is used
Uses dense layers after convolution to combine image features into ImageNet class predictions.
How deep it goes
Explains dense connectivity, flattening, parameter concentration, and output scoring, but gives no weight-level calculations.
Examples
Uses AlexNet’s three fully connected layers, 4,096-unit hidden layers, 1,000-way output, and 50% dropout in FC6 and FC7.
What you can do after
Understand how dense layers aggregate image features into classification scores and probabilities.

Read how Sutskever's List explains linear layer →

Evolutionary Deep Learning: Genetic algorithms and neural networks

A dense layer is a neural-network layer in which each unit receives inputs from all units in the preceding layer.

How it is used
Connects flattened CNN features to class outputs and represents dense-layer choices in evolutionary architecture encodings.
How deep it goes
Shows Keras construction and neuron-count encoding, but does not derive the layer's mathematical internals.
Examples
Uses a Fashion-MNIST CNN head with 128 hidden neurons and 10 outputs, plus a gene sequence encoding a 25-neuron dense layer.
What you can do after
Build it from scratch via Keras and encode its neuron count for generated CNN architectures.

Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains linear layer →

Transformers in Action

nn.Linear is a PyTorch module that applies an affine linear transformation to its input.

How it is used
Transforms fixed-length vectors into representations suitable for feed-forward networks, transformer embeddings, and prediction heads.
How deep it goes
Explains full connectivity, dimensional expansion and contraction, nonlinear placement, and independent reuse across sequence positions.
Examples
Uses a position-wise FFN, a vision-transformer patch projection, and a masked-token prediction head.
What you can do after
Build it from scratch as a PyTorch module inside an FFN or as a projection from patches or hidden states.

Read how Transformers in Action explains linear layer →

Machines that Think

A linear layer is a neural-network layer that applies a learned linear transformation to its inputs.

How it is used
Uses a linear layer to transform contextualized Transformer representations before token probabilities are assigned.
How deep it goes
Explains output prediction and attention projections at an overview level, without specifying matrix shapes or implementation details.
Examples
Illustrates learned projections with attention to “sun” and “east” in a sentence about sunrise.
What you can do after
Evaluate how learned transformations contribute to attention and next-token prediction.

Read how Machines that Think explains linear layer →

Build a Text-to-Image Generator (from Scratch)

A fully connected layer is a neural-network layer in which every output unit receives input from every unit in the preceding layer.

How it is used
Uses learned projections, feed-forward transformations, and classifier heads within attention, transformer, and image-classification architectures.
How deep it goes
Explains placements, dimensions, and surrounding operations, but does not derive parameter gradients or matrix multiplication internals.
Examples
Demonstrates attention projections, a 48-to-192-to-48 transformer MLP, ten-class ViT logits, and 1,000-class ResNet50 logits.
What you can do after
Builds linear transformations in PyTorch using nn.Linear and places them within attention, MLP, or classification components.

Read how Build a Text-to-Image Generator (from Scratch) explains linear layer →

Grokking AI Algorithms, Second Edition

A linear layer is a neural-network layer that computes an affine or linear transformation of its input using learned weights and usually a bias.

How it is used
Transforms token representations inside feed-forward blocks and maps final outputs to vocabulary scores for next-token prediction.
How deep it goes
Traces matrix multiplication, bias addition, ReLU, logits, softmax, and argmax, but avoids deeper parameter-learning internals.
Examples
Uses NumPy projection code, sample weight tables, token vectors, a logit calculation, and vocabulary probability tables.
What you can do after
Build a simple NumPy projection and interpret its logits as scores for selecting a next token.

Read how Grokking AI Algorithms, Second Edition explains linear layer →

CUDA for LLMs

A linear layer is a neural-network layer that computes an affine transformation of its inputs using learned weights and a bias.

How it is used
Computes neural-network layer outputs and supplies matrix operations used during training.
How deep it goes
Derives forward and backward matrix formulas, shapes, bias broadcasting, and input caching without detailing CUDA implementation.
Examples
Uses symbolic batch and feature dimensions to demonstrate output shapes and weight, input, and bias gradients.
What you can do after
Understand how to compute a linear layer’s forward output and training gradients, including which input to cache.

Read how CUDA for LLMs explains linear layer →

Also covered in

What to know first

Related concepts

Questions about linear layer

What is linear layer?

A linear layer is a neural-network layer that applies a learned affine transformation to its input, typically using weights and a bias.

Which Manning books explain linear layer?

Inside Deep Learning: Math, Algorithms, Models, Build AI Drug Discovery Pipelines, Deep Learning with Python, Third Edition, Deep Learning with PyTorch, Second Edition, Deep Learning with R, Third Edition, Math and Architectures of Deep Learning, Grokking Machine Learning, Natural Language Processing in Action, Second Edition and 9 more.

What should I know before learning linear layer?

Manning books that teach linear layer usually introduce embedding, objective function, language model and classifier first.

What is linear layer part of?

Linear layer is part of feedforward network. A feedforward network is a neural network whose computations proceed from inputs toward outputs through directed connections without recurrent cycles or feedback.

What is linear layer also called?

Linear layer is also called fully connected layer, Dense layer, nn.Linear, densely connected layers or fc.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.