concept grad attribute in category deep learning
appears as: grad attribute

This is an excerpt from Manning's book Deep Learning with PyTorch.
Notice the
requires_grad=Trueargument to the tensor constructor? That argument is telling PyTorch to track the entire family tree of tensors resulting from operations onparams. In other words, any tensor that will haveparamsas an ancestor will have access to the chain of functions that were called to get fromparamsto that tensor. In case these functions are differentiable (and most PyTorch tensor operations will be), the value of the derivative will be automatically populated as agradattribute of theparamstensor.