A loss surface represents how a neural network’s loss changes across possible values of its weights and biases. Even for a modest network, it has too many dimensions to visualize directly. Neural-network loss surfaces are generally non-convex, with valleys, flat regions, saddle points, and potentially multiple local minima. section 11.6.1
Gradient descent follows the surface’s local slope, but is not guaranteed to reach the global minimum. Its result can depend on initial parameters, mini-batch order, optimization method, and learning rate. The deepest point in training loss is not necessarily the best choice: the practical aim is good performance on unseen observations. section 11.6.1


