A neuron is a small processing unit in a neural network. It receives several values, gives each a different influence through a weight, combines them, and transforms the result into an activation. That activation can become input to another neuron or contribute to a prediction. section 9.2
The idea is loosely inspired by biological neurons: dendrites receive signals, synapses affect them, and an axon passes a signal onward. An artificial neuron is a simpler calculation, not a miniature brain. A Perceptron represents one such unit; an artificial neural network connects many units so they can work on problems with more input features. section 9.1 section 9.2 section 9.3
To follow one neuron’s calculation:
- Start with input values, such as a car’s speed and the driver’s degree of vision. section 9.3
- Multiply each input by its connection weight. The weights determine how strongly the inputs influence this neuron. section 9.2
- Add the weighted values, then apply an activation function to produce the neuron’s output. section 9.2
From driving conditions to a collision prediction
Imagine predicting whether a collision will occur from speed, terrain quality, vision, and driving experience. Scale the features so their different numerical ranges do not distort their influence. A hidden neuron combines weighted feature values and activates; several hidden neurons do this before an output node produces the prediction. With trained weights, this sequence is called forward propagation. Training adjusts the weights; a prediction uses the weights already learned. section 9.3 section 9.4 section 9.6
The same calculation need not be written separately for every neuron. Put the connection weights for a layer into a matrix, and matrix multiplication calculates its neurons’ weighted sums together. A learning-oriented Python implementation then applies the sigmoid activation to the hidden layer and, similarly, to the output layer. section 9.21 section 9.4
class NeuralNetwork:
def __init__(self, features, labels, hidden_node_count):
num_features = features.shape[1]
num_samples = features.shape[0]
num_output = labels.shape[1]
self.input = features
self.expected_output = labels
self.hidden_node_count = hidden_node_count
self.weights_input =
➥np.random.randn(num_features, hidden_node_count) * 0.01
self.weights_hidden =
➥np.random.randn(hidden_node_count, num_output) * 0.01
self.hidden = np.zeros((num_samples, hidden_node_count))
self.output = np.zeros((num_samples, num_output))
def forward_propagation(self):
self.hidden = sigmoid(np.dot(self.input, self.weights_input)) #A
self.output = sigmoid(np.dot(self.hidden, self.weights_hidden)) #B
Inspect how two dot products and sigmoid activations carry values through the hidden and output layers. section 9.4
