A gradient tells you how a function’s output changes when its tensor input changes. In neural-network training, the function maps model weights to a loss value. Its gradient points toward increasing loss, so moving the weights a little in the opposite direction can reduce that loss. This is more efficient than testing changes to every weight separately. section 2.4.2 section 2.4
A gradient for model weights
Start with an input vector x, a weight matrix W, and a target y_true. Compute y_pred from x and W, then measure its mismatch with y_true using a loss function. With x and y_true fixed, the loss is a function of W. The gradient has the same shape as W: each entry indicates the direction and magnitude of the loss change associated with changing that weight. section 2.4.2
one_training_step <- function(model, images_batch, labels_batch) {
with(tf$GradientTape() %as% tape, {
predictions <- model$call(images_batch)
per_sample_losses <-
op_sparse_categorical_crossentropy(labels_batch, predictions)
average_loss <- op_mean(per_sample_losses)
})
gradients <- tape$gradient(average_loss, model$weights)
update_weights(gradients, model$weights)
average_loss
}
In this R training step, tf$GradientTape records the prediction and loss operations. tape$gradient() then obtains the average loss gradient for the model weights, which update_weights() uses to change them. section 2.5.2
For each mini-batch, gradient-based training follows a short cycle:
- Run a forward pass and calculate the loss against the batch’s targets. section 2.4.3
- Compute the loss gradient with respect to the weights in a backward pass. section 2.4.3
- Update the weights in the opposite direction, scaled by a learning rate. A rate that is too small makes progress slow; one that is too large can make updates erratic. section 2.4.3
Gradients are useful beyond weight training. PyTorch can obtain a gradient with respect to an input image and use it for gradient ascent. Gradients can also be pooled and weighted to produce a class activation heatmap. section 10.2.2 section 10.3.4
