Gradient Ascent

Gradient ascent is an optimization algorithm used to find the maximum value of a function by iteratively moving in the direction of the gradient. The gradient indicates the direction of the steepest increase, making it a powerful tool for maximizing functions.

Overview

The core idea of gradient ascent is to find the direction in which a function increases most quickly by using the gradient. By moving in this direction, one can approach the maximum value of the function. This involves calculating the partial derivatives of the function with respect to its variables, which together form the gradient vector.

Implementing Gradient Ascent

The implementation of gradient ascent involves starting from an initial point and iteratively updating the point by moving in the direction of the gradient. The process continues until the gradient’s magnitude falls below a specified tolerance, indicating that a local maximum has been reached. Below is a Python implementation of the gradient ascent algorithm:

def gradient_ascent(f, xstart, ystart, tolerance=1e-6):
    x = xstart
    y = ystart
    grad = approx_gradient(f, x, y)
    while length(grad) > tolerance:
        x += grad[0]
        y += grad[1]
        grad = approx_gradient(f, x, y)
    return x, y

Example

Consider the following example where the gradient_ascent function is used to find the maximum value of a function landing_distance starting from the point (36, 83):

>>> gradient_ascent(landing_distance, 36, 83)
(37.58114751557887, 89.99992616039857)

This result indicates that the maximum value is reached at approximately (37.58, 90.00).

Visualizing Gradient Ascent

To better understand how the algorithm works, we can visualize the trajectory of the gradient ascent through the θ, ϕ plane. The following figures illustrate the process:

Figure 12.27 The starting and ending points for the gradient ascent

Figure 12.28 The path that the gradient ascent algorithm takes to reach the maximum value of the range function

The path taken by the algorithm is shown in Figure 12.28, where the algorithm took 855 steps to complete the ascent. The path and the destination depend on the choice of the initial point. If the starting point is close to ϕ = 90°, the algorithm is likely to find that maximum. Conversely, if the starting point is closer to ϕ = 270°, the algorithm will find that maximum instead, as shown in Figure 12.29.

Figure 12.29 Starting at different points, the gradient ascent algorithm can find different maximum values.

The launch angles (37.58°, 90°) and (37.58°, 270°) both maximize the function r(θ, ϕ) and yield the greatest range for the cannon, which is about 53 meters.