A learning rate, η, sets how far parameters move in response to their gradients during gradient descent. It is selected before training, not calculated by the network. section 11.3.5
The update subtracts the gradient scaled by η from the current parameter. section 11.3.5
One update
With η = 0.1, a parameter at 1.2 with gradient −0.24 becomes 1.224. A negative gradient increases the parameter; a positive gradient decreases it. The example updates seven network parameters from one observation. section 11.3.5






















