Batch gradient descent updates model parameters by computing the gradient from the entire training dataset. Its steps become smaller as the cost approaches a minimum, with learning rate controlling convergence behavior. section 4.2.3
Practical training requires monitoring several factors:
- Combine a large iteration limit, an acceptable cost threshold, and a marginal-cost-decrease threshold to decide when to stop. section 4.2.3
- Compare batch updates with stochastic and mini-batch methods, which reduce per-update data requirements but produce noisier convergence. section 4.2.3 section 4.5
- Scale features when different variable magnitudes would cause some variables to dominate training. section 4.2.3
Molecular solubility
A linear model trained on AqSolDB can use molecular descriptors to characterize solubility, a drug-discovery property linked to absorption and dosing. Batch gradient descent provides an iterative alternative to the normal equation for finding model weights. section 4.5
Figure 4.11 Impact of learning step size on the ability for gradient descent to converge.

Use the convergence illustration to inspect how learning-step size affects whether gradient descent reaches a minimum. section 4.2.3

