Linear layer

As explained in Grokking Machine Learning

A neural-network architecture in which every node in a layer connects to every non-bias node in the next layer, with no connections between nonconsecutive layers.

A linear layer connects every node to every non-bias node in the next layer. It skips no nodes within that adjacent layer, and connections do not jump across layers. Bias nodes are part of the architecture but are not counted in a layer’s size. section 10.1

A neural network arranges nodes into an input layer, hidden layers, and an output layer. Its depth counts layers after the input layer. For example, an input layer of size 4, hidden layers of sizes 5 and 3, and an output layer of size 1 have depth 3. section 10.1

Figure 10.14 The general architecture of a neural network. The nodes are divided into layers, where the leftmost layer is the input layer, the rightmost layer is the output layer, and all the layers in between are hidden layers. All the nodes in a layer are connected to all the (non-bias) nodes in the next layer.

See how each layer connects to the next in a network diagram. section 10.1

How Grokking Machine Learning covers it

How deep it goes

Defines layer connectivity, layer sizes, and depth, while pointing to other architectures without detailing them.

Examples

Uses a network with layer sizes 4, 5, 3, and 1 to illustrate depth.

What you can do after

Understand how to identify layer connections and count network depth.

How it connects

Connects neural networks with input, hidden, and output layers, depth, and bias nodes.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA PyTorch module that applies an affine linear transformation to its input.
• 

  liveBookmanning.com
  Deep Learning with Python, Third Edition
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionA PyTorch linear layer module that applies an affine transformation to map features to outputs.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA Dense layer is a neural-network layer in which every output unit is connected to every input unit through learned weights and biases.
• 

  liveBookmanning.com
  Math and Architectures of Deep Learning
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA fully connected linear layer is a neural-network layer that computes an affine transformation from every input activation to its output units.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA PyTorch layer that applies a learned linear transformation to its input.
• 

  liveBookmanning.com
  Sutskever's List
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksA dense layer is a neural-network layer in which each unit receives inputs from all units in the preceding layer.
• 

  liveBookmanning.com
  Transformers in Actionnn.Linear is a PyTorch module that applies an affine linear transformation to its input.
• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)A fully connected layer is a neural-network layer in which every output unit receives input from every unit in the preceding layer.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA linear layer is a neural-network layer that computes an affine or linear transformation of its input using learned weights and usually a bias.

Compare how 13 books explain linear layer →

Related concepts

• affine transformimplements · An affine transformation is a mapping that combines a linear transformation with an additive translation or offset.
• feedforward networkpart of · A feedforward network is a neural network or subnetwork in which information flows from inputs toward outputs through successive computations without recurrent connections or cycles.
• logitproduces · A logit is a real-valued quantity representing an outcome’s log-odds or, in machine learning, an unnormalized model score used to derive probabilities.
• matrix-matrix multiplicationuses · Matrix multiplication combines compatible matrices to form a result whose entries are sums of products between rows of the first matrix and columns of the second.
• softmaxuses · Softmax is a function that exponentiates a vector of scores and normalizes the results into nonnegative values that sum to one.
• neural network layerkind of · A neural network layer is a computational stage or component that transforms input representations into outputs within a neural network.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.