Gradient clipping limits gradient values during gradient descent so an exploding gradient does not push model weights too far across the cost-function landscape. It is useful when training models for solubility prediction, where unstable updates can disrupt convergence. section 4.2.3
Apply the technique by:
- Identifying a gradient cliff: a steep drop in the cost function that produces an unusually large gradient. section 4.2.3
- Setting minimum and maximum clip values, then preventing each gradient from falling below or exceeding those bounds. section 4.2.3
- Considering learning rate, convergence criteria, and feature scaling alongside clipping when diagnosing gradient-descent behavior. section 4.5 section 4.2.3
Figure 4.13 Gradient cliffs impact converging to an optimal minimum of model weights.

Use the figure to see how gradient cliffs interfere with reaching an optimal minimum of the model weights. section 4.2.3


