Gradient descent iteratively minimizes a differentiable cost function by calculating how each parameter affects the cost and moving parameters along the negative gradient. The learning rate controls step size: updates should be substantial far from a minimum and smaller near one. section 4.2.3
Practical choices shape optimization:
- Batch gradient descent uses the full training set; stochastic and mini-batch variants update from individual examples or small batches, trading memory and runtime against noisier convergence. section 4.2.3 section 4.5
- Combine an iteration limit, an acceptable cost threshold, and a marginal-improvement threshold to avoid stopping too early on a plateau. section 4.2.3
- Learning rate, gradient clipping, and feature scaling affect stability and convergence; convex linear-regression and logistic-regression objectives have a global minimum under suitable updates. section 4.2.3 section 5.1.1
Solubility and inhibition prediction
For molecular solubility, gradient descent finds linear-model weights when using the mean-squared-error objective. For cytochrome P450 inhibition, it iteratively minimizes binary cross-entropy after the model converts scores into probabilities with the sigmoid function. section 4.5 section 5.1.1
Figure 4.10 Example cost function surface we may navigate with gradient descent.

A cost-function surface makes the high-dimensional downhill search intuitive; related figures show step-size effects and stopping behavior. section 4.2.3























