A validation set is held-out data used to assess a model during training. Its loss provides a signal about generalization and overfitting without updating the model’s weights. section 13.4.2
A validation loop supports training decisions in two ways:
- Evaluate the model on validation batches and average their loss. section 13.4.2
- Track whether validation loss improves; stop after three consecutive epochs without improvement, and retain the best weights. section 13.4.2
def val_epoch(model,dataloader,loss_fn): #1
model.eval()
val_loss=0
with torch.inference_mode():
for batch, (X, y) in enumerate(dataloader): #2
X, y = X.to(device), y.to(device)
pred_logits = model(X)
loss = loss_fn(pred_logits, y)
val_loss += loss.item()
return val_loss/len(dataloader) #3
The evaluation function switches the model to evaluation mode, disables gradient tracking, and accumulates loss across validation batches. section 13.4.2
