liveBook · Manning

Gradient ascent

As explained in Applied Reinforcement Learning

An iterative optimization method that updates parameters in the direction of a function's gradient to increase its value.

Gradient ascent improves a policy by moving its parameters uphill toward higher reward. The gradient points toward faster improvement, while the learning rate controls the step size. section 9.2

For policy parameters θ, the objective J(θ) is expected cumulative reward. The update θ ← θ + α∇θJ(θ) moves the parameters toward higher expected return—the reward-maximizing counterpart to gradient descent. section 9.2

Figure 9.2 The Policy Gradient Objective.

See the policy-gradient objective represented visually. section 9.2

How Applied Reinforcement Learning covers it

How deep it goes
Defines the expected-return objective and parameter update, then identifies trajectory expectations as a gradient-computation challenge.
What it assumes
Introduces policy parameters, trajectories, and expected return before applying the gradient update.
What you can do after
Understand how to move policy parameters toward higher expected return using a gradient and learning rate.
How it connects
Connects policy gradients with gradient descent, supervised learning, and DQN parameter updates.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.