Training fits a model's parameters Θ by minimizing loss over labeled examples. With inputs x, labels y, and predictions ŷ = fΘ(x), the objective is minΘ Σᵢ ℓ(fΘ(xᵢ), yᵢ). section 2.1.1
Gradient descent changes Θ to reduce the loss; PyTorch tracks tensor operations and computes the gradients needed for this update. section 2.1.1 section 1.7
def F(X, y, f, theta):
total_loss = 0
for i in range(N):
total_loss += loss(f(X[i,:], theta), y[i])
return total_loss
A compact loop-style listing maps the summation in the training objective to accumulated loss over input–label pairs. section 2.1.1
Training choices can be tested empirically:
- Run CNNs for 40 rather than 10 epochs and observe the effect. section 3.7
- Train CIFAR10 models while varying convolution layers, pooling, filters, and image transforms. section 3.7
