A weight is a numerical parameter that controls how strongly an input contributes to a neuron's output. Think of it as sensitivity: increasing a weight makes a given input matter more. In a simple perceptron, each numerical feature is multiplied by its weight, and the products are added to make a prediction or decision. section 5.1.3 section 5.1.5
From word counts to a prediction
Suppose a text becomes the word-count vector [1, 2, 3, 4]. Give those counts weights [0.12, −0.16, 0.03, −0.18] and add a bias of 0.07. The weighted sum is −0.76. A threshold at zero turns that score into a binary decision. The weights, rather than the counts themselves, determine how each word affects the score. section 5.1.5
>>> from collections import Counter >>> np.random.seed(451) >>> tokens = "green egg egg ham ham ham spam spam spam spam".split() >>> bow = Counter(tokens) >>> x = pd.Series(bow) >>> x green 1 egg 2 ham 3 spam 4 >>> x1, x2, x3, x4 = x >>> x1, x2, x3, x4 (1, 2, 3, 4) >>> w0 = np.round(.1 * np.random.randn(), 2) >>> w0 0.07 >>> w1, w2, w3, w4 = (.1 * np.random.randn(len(x))).round(2) >>> w1, w2, w3, w4 (0.12, -0.16, 0.03, -0.18) >>> x = np.array([1, x1, x2, x3, x4]) #1 >>> w = np.array([w0, w1, w2, w3, w4]) #2 >>> y = np.sum(w * x) #3 >>> y -0.76
Inspect the Python calculation to see word counts and weights assembled into arrays before their products are summed. section 5.1.5
Learning means changing the weights to reduce prediction error. Picture error as the height of a landscape whose horizontal coordinates are possible weights. Gradient descent adjusts those coordinates toward lower error; a learning rate controls the size of each adjustment. The landscape need not be a smooth bowl, so training can settle in a local minimum. Monitoring error over time helps show whether training is making progress. section 5.2.5 section 5.3 section 5.3.1
In PyTorch, a linear layer holds trainable parameters. A training loop clears old gradients, computes predictions and loss, calls `loss.backward()`, then calls `optimizer.step()` to update parameters. For a recurrent neural network, the same weight matrices are used across tokens; backpropagation through time carries error backward across those uses. More parameters increase model capacity, but size alone is not proof of understanding or reliable answers. section 5.2.5 section 8.3 section 10.1.1
