A neuron is the smallest computing unit in an artificial neural network. It receives a vector of input signals, combines them with its weights using a dot product, then applies an activation function to produce one output. Without nonlinear activation functions, a network of these units would amount to a linear transformation. section 7.2
Figure 7.2. A single neuron combines its weights with input signals to produce an output signal that is modified by an activation function.
Follow one input vector through the weighted combination and activation function to see where the output signal comes from. section 7.2
In a feed-forward network, neurons are grouped into layers. A layer passes its outputs to the next layer. The input layer simply passes its supplied values onward; neurons in hidden and output layers compute outputs from those values. section 7.2 section 7.5
A practical Swift `Neuron` keeps the state needed for both prediction and training:
- `weights: [Double]` and an activation function determine its output. `output(inputs:)` caches the dot product before applying activation. section 7.5
- An activation-function derivative, `delta`, and a learning rate support weight updates. Output-layer deltas use expected versus actual output; hidden-layer deltas use the next layer’s weights and deltas. section 7.5
// should only be called on an output layer
func calculateDeltasForOutputLayer(expected: [Double]) {
for n in 0..<neurons.count {
neurons[n].delta =
neurons[n].derivativeActivationFunction(neurons[n]
.outputCache) * (expected[n] - outputCache[n])
}
}
// should not be called on output layer
func calculateDeltasForHiddenLayer(nextLayer: Layer) {
for (index, neuron) in neurons.enumerated() {
let nextWeights = nextLayer.neurons.map { $0.weights[index] }
let nextDeltas = nextLayer.neurons.map { $0.delta }
let sumOfWeightsXDeltas = dotProduct(nextWeights, nextDeltas)
neuron.delta = neuron.derivativeActivationFunction(neuron
.outputCache) * sumOfWeightsXDeltas
}
}
}
Compare how output-layer and hidden-layer neurons calculate their deltas before weights are updated. section 7.5
The sigmoid activation function gives each neuron an output between 0 and 1. Its computable derivative makes it usable during backpropagation. Training compares network outputs with expected values, calculates deltas, then adjusts weights using those deltas, prior-layer outputs, and the learning rate. section 7.4 section 7.5
Classifying irises
A `Network(layerStructure: [4, 6, 3], learningRate: 0.3)` accepts four measurements, uses six hidden-layer neurons, and produces three class outputs. After feature scaling, training uses 140 iris samples and testing uses the remaining 10; the shown run classifies 9 of 10 correctly. section 7.6
