Suppose each extra room adds $50 to a house’s predicted price, but even the starting price is $100. That $100 is the model’s bias: a constant added to the feature-based part of a prediction. Here, “bias” names a number in a formula, not an unfair outcome. section 3.2
Price = 100 + 50(Number of rooms)
In Price = 100 + 50(Number of rooms), 50 is the weight on the number of rooms; 100 is the bias. The formula predicts $300 for four rooms. section 3.2
In linear regression, each feature has a weight, and the weighted values are added together with the bias. On a graph with one feature, changing the weight changes the line’s slope; changing the bias moves the line up or down. A training procedure can start with random weights and bias, then repeatedly adjust them to improve predictions for selected data points. section 3.9 section 3.3
A bias that sets a classification threshold
Imagine classifying a sentence by giving each word one point. If sentences of three words or fewer are sad and those of four or more are happy, a decision boundary is #crack + #doink − 3.5 = 0. The −3.5 is the bias: it sets the cutoff even though neither word contributes it. More generally, a perceptron adds weighted features and a bias, then predicts one category when the score is at least zero and the other when it is negative. With no words present, the bias alone determines the prediction. section 5.1 section 5.6
There is another meaning to keep separate: bias in data can reflect historical social inequalities. A historical passenger dataset, for example, records gender and class as they were represented and treated at the time; it cannot be resampled to reflect current norms. A mathematically useful bias term does not resolve such problems. When working with historical data, consider whether a model could perpetuate the biases it contains. section 13.1
Try changing only the $100 bias in the house-price formula. Which way does the prediction line move, and what happens to the four-room prediction? Then consider the word-count classifier: what would changing only −3.5 do to its cutoff? section 3.2 section 5.1
