Suppose each extra room adds $50 to a house’s predicted price, on top of a $100 base price. The $50 is a weight: a number that multiplies the feature “number of rooms.” The $100 is a bias: a constant added whether the house has many rooms or few. Both are model parameters—numbers the model can change during training. section 3.2 section 3.7
One weight, one prediction
Predicted price = 100 + 50 × number of rooms. For four rooms, the prediction is $300. Here the weight controls how much the prediction changes per room; the bias supplies the base price. section 3.2
With several features, give each one its own weight, multiply each feature by its weight, and add the results and the bias. A house-price model might use room count, size, school quality, and age. Its weights determine how each of those features contributes to the predicted price. For a single feature, changing the weight changes the slope of the model’s line; changing the bias moves the line up or down. section 3.2 section 3.3
Training finds useful parameter values rather than requiring you to guess them perfectly. A simple linear regression procedure starts with a random weight and bias, picks a data point, and adjusts them slightly to improve that point’s prediction. It repeats this many times, moving the line toward the observed prices. A better prediction for one point does not, by itself, establish a better model for every point. section 3.3
Weights also appear in classifiers. A perceptron adds weighted features and a bias to produce a score: a score of zero or more predicts one category, and a negative score predicts the other. For word-count features, a positive weight can favor a happy classification while a negative weight favors a sad one. The weights and bias together determine the line that separates the two categories. section 5.1 section 5.6
Inspect weights in movie reviews
A Turi Create logistic classifier trained on IMDB review word counts gives “wonderful” a weight of 1.043, “horrible” −1.075, and “the” about 0.0005. Inspecting the coefficients shows which words push the model toward positive or negative sentiment; the intercept is its bias. section 6.4
Do not confuse a parameter with a hyperparameter. The model creates or modifies parameters during training; you set hyperparameters, such as the learning rate or number of epochs, before training. To see the distinction in action, try changing the learning rate in a small housing-price regression and watch how its learned weight and bias change. section 3.7 section 3.3
