liveBook · Manning

Gradient descent

As explained in Build AI Drug Discovery Pipelines

Gradient descent is an iterative optimization method that updates parameters in the direction that decreases a differentiable objective function.

Gradient descent iteratively minimizes a differentiable cost function by calculating how each parameter affects the cost and moving parameters along the negative gradient. The learning rate controls step size: updates should be substantial far from a minimum and smaller near one. section 4.2.3

Practical choices shape optimization:

  • Batch gradient descent uses the full training set; stochastic and mini-batch variants update from individual examples or small batches, trading memory and runtime against noisier convergence. section 4.2.3 section 4.5
  • Combine an iteration limit, an acceptable cost threshold, and a marginal-improvement threshold to avoid stopping too early on a plateau. section 4.2.3
  • Learning rate, gradient clipping, and feature scaling affect stability and convergence; convex linear-regression and logistic-regression objectives have a global minimum under suitable updates. section 4.2.3 section 5.1.1

Solubility and inhibition prediction

For molecular solubility, gradient descent finds linear-model weights when using the mean-squared-error objective. For cytochrome P450 inhibition, it iteratively minimizes binary cross-entropy after the model converts scores into probabilities with the sigmoid function. section 4.5 section 5.1.1

Figure 4.10 Example cost function surface we may navigate with gradient descent.

A cost-function surface makes the high-dimensional downhill search intuitive; related figures show step-size effects and stopping behavior. section 4.2.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains gradients, updates, learning rates, variants, stopping criteria, clipping, and scaling without deriving optimizer implementations.
Examples
Uses AqSolDB solubility modeling and cytochrome P450 inhibition classification as applied examples.
What it assumes
Assumes parameters, differentiable cost functions, gradients, and model weights, while motivating them through linear and logistic prediction.
What you can do after
Apply it via batch, mini-batch, or stochastic updates while selecting step sizes and termination criteria for stable convergence.
How it connects
Connects linear regression, logistic regression, mean squared error, binary cross-entropy, stochastic gradient descent, learning rate, and feature scaling.

Also explained in

Compare how 24 books explain gradient descent →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.