A hyperparameter is chosen outside the model’s learned weights and biases to control how learning proceeds. The learning rate sets the size of each gradient-descent update: large steps can overshoot, while small steps can make progress impractically slow. section 3.2.3
Figure 3.8 The loss landscape can contain many dips and valleys. If the optimizer steps too cautiously, it may settle into a shallow solution with relatively high loss. If it steps too boldly, it may skip over configurations that yield lower loss or bounce back and forth across a good solution without settling. In practice, these landscapes exist in very high-dimensional spaces, where their geometry becomes more complex. Finding the right step size is key to guiding learning toward regions of lower loss.

The loss-landscape illustration contrasts cautious steps that may settle at higher loss with bold steps that may skip or bounce past better configurations. section 3.2.3

























