A neural network layer groups neurons so their outputs can become inputs to the next group. In a feed-forward network, signals move one way: from the input layer, through hidden layers, to the output layer. Every neuron in one layer connects to every neuron in the next. The signals are floating-point numbers; the final numbers need interpretation to become a result. section 7.2
Figure 7.3. A simple neural network with one input layer of two neurons, one hidden layer of four neurons, and one output layer of three neurons. The number of neurons in each layer in this figure is arbitrary.
The diagram shows two input neurons, four hidden neurons, and three output neurons. Those counts illustrate a possible layout, not required layer sizes. section 7.2
A small Swift implementation gives each layer three jobs:
- Hold its neurons, a reference to the previous layer, and a cached array of outputs. section 7.5
- Pass inputs straight through if it is the input layer; otherwise, call each neuron's output method and cache the resulting values. section 7.5
- During training, calculate neuron deltas from expected outputs or from the next layer's weights and deltas. Weight updates then use the previous layer's cached outputs. section 7.5
Passing signals forward
A network can pass an array of Double values through its layers with `layers.reduce(input) { $1.outputs(inputs: $0) }`. Each call receives the preceding layer's output array; the last call returns the network's output. section 7.5
A neuron's output combines its inputs and weights, then applies an activation function before reaching the next layer. The sigmoid function is one available choice: it keeps each resulting value between 0 and 1, and its derivative can be used in backpropagation. section 7.5 section 7.4
