Backpropagation computes how the loss changes with respect to model parameters, then supplies gradients that an optimizer uses to update the model. In a typical training loop, it follows the forward pass and loss calculation, making it the step where learning occurs. section 2.2.1 section 2.3.2
A PyTorch training step separates gradient calculation from parameter updates:
- Run the model on a batch and calculate the loss. section 3.4.1
- Call zero_grad() before backward() so each batch uses incremental rather than accumulated gradients. section 3.4.1
- Call loss.backward() to compute gradients for the model parameters. section 2.3.2
- Call optimizer.step() to adjust parameters using those gradients. section 2.3.2
Training a binary classifier
For a binary clothing classifier, the loop feeds flattened images through the network, computes binary cross-entropy, clears old gradients, backpropagates with loss.backward(), and updates the model with optimizer.step(). The same pattern trains both discriminator and generator networks in a GAN. section 2.3.2 section 3.4.1
Figure 2.1 The steps involved in training a deep learning model
Use the training-process figure to connect the forward pass, loss calculation, backpropagation, and weight update in sequence. section 2.2.1
