liveBook · Manning

Neural network layer

As explained in Timeless Algorithms, The Foundational Ideas

A layer is a stage in a neural network that transforms values received from a preceding stage.

A neural network layer is a stage that transforms the numerical values it receives. The input layer receives an observation, hidden layers build successive internal representations, and the output layer produces the task’s prediction. section 11.2.1

A node in a layer transforms its incoming values through a sequence of operations:

  • Multiply each input by its weight, sum the results, and add a bias: z = w₁x₁ + w₂x₂ + … + wₙxₙ + b. section 11.2.2
  • Apply an activation function, a = ϕ(z), then pass the result to the next layer. The activation introduces nonlinearity. section 11.2.1 section 11.2.2

A layer’s outputs form a representation of its input, not a copy of the original data. Successive hidden layers can reorganize values so the distinctions relevant to prediction become easier for the output layer to separate. section 11.2.1 section 11.5.2

Figure 11.3 A nonlinear transformation reorganizes the same six observations so that classes requiring a curved boundary in the input space can be separated by a straight boundary in the learned representation.

The diagram traces six observations from a space where classes need a curved boundary to a learned representation where a straight boundary separates them. section 11.5.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Works through weighted sums, biases, activation functions, and representation changes, without detailing specific layer designs.
Examples
Uses image features and six circles-and-squares observations to illustrate how layers reorganize data for classification.
What you can do after
Understand how successive layer transformations can make observations easier to classify.
How it connects
Links layers to neural network architecture, forward passes, activation functions, and learned representations.

Also explained in

Compare how 12 books explain neural network layer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.