A learning rate is a small positive number chosen before training. It controls how far a model moves when it updates its parameters, so learning happens in small steps rather than sudden jumps. section 3.13 section 10.16
Moving a house-price line
Suppose a line predicts house prices from the number of rooms. If the actual price p is above the prediction p̂, the difference p − p̂ is positive, so the line moves upward. Multiplying that difference by the learning rate keeps the adjustment small. The same idea updates the slope using h times the number of rooms and the prediction error. section 3.13
def square_trick(base_price, price_per_room, num_rooms, price, learning_rate):
predicted_price = base_price + price_per_room*num_rooms #A
base_price += learning_rate*(price-predicted_price) #B
price_per_room += learning_rate*num_rooms*(price-predicted_price) #C
return price_per_room, base_price
See the learning rate applied separately to the base price and price-per-room parameter in a small update function. section 3.13
A learning rate works with repeated updates. In linear regression, many small steps gradually move a line closer to the data; in a perceptron, it is one of the two main training hyperparameters alongside the number of epochs. section 3.17 section 5.15
The learning rate appears in several update rules:
- Square trick: add h(y − ŷ) to the bias and hxi(y − ŷ) to each weight. section 3.19
- Absolute trick: add or subtract h and h times the feature, depending on whether the point lies above or below the line. section 3.14
- Neural networks: use the learning rate as the size of each training step and tune it with other hyperparameters. section 10.16






















