PyTorch is a Python deep learning library built around tensors: multidimensional arrays that represent data, network inputs and outputs, and learnable parameters. Tensor operations also support GPU execution and automatic differentiation, providing the computational foundation for neural network training. section 1.2 section 1.7
Use PyTorch to:
- Represent scalars, vectors, matrices, and higher-dimensional arrays with torch.tensor; inspect their shapes and slice sub-tensors. section 1.2
- Move tensor calculations to a CUDA device when the workload is large enough to offset CPU-to-GPU transfer costs. section 1.2.1
- Track tensor operations and compute derivatives automatically instead of deriving every gradient by hand. section 1.3.1 section 2.1.1
- Minimize a loss by updating parameters in the opposite direction of their gradients through an optimizer such as stochastic gradient descent. section 1.4
A minimal optimization loop
For f(x) = (x − 2)², make x a learnable parameter, calculate the loss, backpropagate, and update it repeatedly: x_param = torch.nn.Parameter(torch.tensor([-3.5]), requires_grad=True); optimizer = torch.optim.SGD([x_param], lr=0.1). Each iteration calls optimizer.zero_grad(), evaluates f(x_param), calls loss.backward(), and finishes with optimizer.step(). The parameter approaches 2, the function's minimum. section 1.3.3 section 1.4
Figure 1.4 Examples of tensors, with more dimensions or axes as we move from left to right. A scalar represents a single value. A vector is a list of values and is how we often think about one datapoint. A matrix is a grid of values and is often used for a dataset. A three-dimensional tensor can be used to represent a dataset of sequences.
Compare scalar, vector, matrix, and three-dimensional tensor shapes to connect mathematical axes with the data structures used in PyTorch. section 1.2
The same pattern scales from one variable to neural networks: define a model and loss, let automatic differentiation produce the gradient for every parameter, and update those parameters to reduce the loss. PyTorch modules can also be composed into larger architectures, reuse weights, and reshape data between image and vector representations. section 2.1.1 section 7.1.2
