A computation graph is a directed graph for representing mathematical operations and the data flowing between them. Nodes perform operations; edges carry data, often multidimensional arrays called tensors. This structure makes complex calculations and their gradients manageable during deep-learning training. section 1.1.3
A computation graph supports training by connecting:
- Mathematical operations represented as nodes. section 1.1.3
- Data flowing between operations as edges. section 1.1.3
- Partial-derivative calculations used by backpropagation and gradient descent to update model parameters. section 1.1.3
Dynamic graphs in PyTorch
During model training, PyTorch creates and modifies the computation graph as operations run. Its dynamic structure adapts to varying model architectures and makes debugging simpler, while GPU execution can accelerate the associated calculations compared with CPU execution. section 1.1.3
