PyTorch is an open-source Python machine-learning framework. Use it when you need tensors, automatic differentiation, and control over a training loop. Its package name is `torch`: install it with `pip install torch` and import it with `import torch`. It is widely used in machine-learning research, and Hugging Face provides first-class support for PyTorch models. section 3.4 section 3.4.1 section 3.4.3
Start with a tensor: `torch.ones((2, 2))` creates a 2 × 2 array of ones, while `torch.matmul(a, b)` performs matrix multiplication. Unlike TensorFlow tensors, PyTorch tensors can be assigned to. For trainable state, use `torch.nn.Parameter`; a `torch.nn.Module` can then track those parameters as part of a model. section 3.4.1 section 3.4.2
A gradient in three steps
Create `x = torch.tensor(3.0, requires_grad=True)`, compute `result = torch.square(x)`, then call `result.backward()`. The gradient is available as `x.grad`, which is `tensor(6.)` here. Watch out: another backward call adds to the stored gradient. Reset it before the next training step. section 3.4.1
For a small linear classifier, the training loop has a practical rhythm:
- Compute predictions and a loss from the inputs and trainable weights. section 3.4.2
- Call `loss.backward()` and read gradients from each parameter’s `.grad`. section 3.4.2
- Update weights inside `torch.no_grad()` so the update is not recorded for backpropagation; then clear the gradients, or they will accumulate across steps. section 3.4.2
Listing 3.27 Defining a torch.nn.Module
class LinearModel(torch.nn.Module): def __init__(self): super().__init__() self.W = torch.nn.Parameter(torch.rand(input_dim, output_dim)) self.b = torch.nn.Parameter(torch.zeros(output_dim)) def forward(self, inputs): return torch.matmul(inputs, self.W) + self.b
This linear model keeps its weights and bias as Parameters and defines its computation in `forward()`—the basic pattern for packaging state and computation in a `torch.nn.Module`. section 3.4.2
You can shorten the loop with `torch.optim.SGD`: after backpropagation, call `optimizer.step()` and `model.zero_grad()`. PyTorch normally executes eagerly, which helps debugging. `torch.compile()` offers a way to try compilation, but the 2025 comparison cautions that its speed benefits and reliability vary. Keras can also use PyTorch as a backend when you want Keras’s higher-level training interface. section 3.4.2 section 3.4.3 section 3.2
