Mean squared error (MSE) measures prediction quality by averaging the squared differences between predicted and target values. Squaring makes larger errors count more heavily, producing a single loss value for optimization. section 6.2.2
MSE fits naturally into gradient-based training:
- Compute predictions from the model and compare them with the targets using MSE. section 6.2.2
- Backpropagate the loss, then let an optimizer such as SGD update the model parameters. section 6.2.2
- Use a low MSE as evidence that predicted values are close to their targets; in diffusion, those values are added noise. section 10.6.1
PyTorch linear regression
Replace a handwritten loss function with nn.MSELoss() in a training loop for a linear model. The model predicts temperatures, MSE reports training and validation error, and SGD updates the weight and bias. section 6.2.2
Training a diffusion model
A denoising model receives a noisy image and timestep, predicts the noise that was added, and uses F.mse_loss(noise, noise_pred) to train accurate noise prediction for later denoising. section 10.6.1
# In[15]:
linear_model = nn.Linear(1, 1)
optimizer = optim.SGD(linear_model.parameters(), lr=1e-2)
training_loop(
n_epochs = 3000,
optimizer = optimizer,
model = linear_model,
loss_fn = nn.MSELoss(), #1
t_u_train = t_un_train,
t_u_val = t_un_val,
t_c_train = t_c_train,
t_c_val = t_c_val)
print()
print(linear_model.weight)
print(linear_model.bias)
# Out[15]:
Epoch 1, Training loss 134.9599, Validation loss 183.1707
Epoch 1000, Training loss 4.8053, Validation loss 4.7307
Epoch 2000, Training loss 3.0285, Validation loss 3.0889
Epoch 3000, Training loss 2.8569, Validation loss 3.9105
Parameter containing:
tensor([[5.4319]], requires_grad=True)
Parameter containing:
tensor([-17.9693], requires_grad=True)
Inspect a complete PyTorch training call that passes nn.MSELoss() into a reusable training loop. section 6.2.2
