Optimization adjusts a model’s internal parameters so its predictions fit training data more closely. A loss or cost function measures the fit, while an optimization algorithm updates parameters iteratively. section 8.2.3 section 4.2.1
A typical training step follows this sequence:
- Reset accumulated gradients with optimizer.zero_grad(). section 8.2.3
- Run a forward pass to produce predictions, then calculate loss against the targets. section 8.2.3
- Call loss.backward() to calculate parameter gradients and optimizer.step() to apply the update. section 8.2.3
- Track training and validation loss across epochs to identify saturation and guide epoch, learning-rate, scheduler, or early-stopping choices. section 8.2.3
Solubility regression
For a solubility model, mean squared error penalizes prediction errors, and optimization searches for weights that minimize this cost. Linear regression can use an analytic solution, while more general neural-network training updates parameters iteratively. section 4.2.1 section 8.2.3
Listing 8.7 Train the Model
def train_model(
model, train_loader, val_loader, criterion, optimizer,
scheduler, device, num_epochs=10
):
model.train() #A
train_losses = [] #B
val_losses = []
for epoch in range(num_epochs): #C
running_loss = 0.0 #D
for inputs, targets in train_loader:
inputs, targets = inputs.to(device), targets.to(device)
optimizer.zero_grad() #E
outputs = model(inputs) #F
loss = criterion(outputs, targets) #G
loss.backward() #H
optimizer.step() #I
running_loss += loss.item()
train_loss = running_loss / len(train_loader)
train_losses.append(train_loss)
model.eval() #J
val_loss = 0.0
with torch.no_grad(): #K
for inputs, targets in val_loader:
inputs, targets = inputs.to(device), targets.to(device)
outputs = model(inputs)
loss = criterion(outputs, targets)
val_loss += loss.item()
val_loss /= len(val_loader)
val_losses.append(val_loss)
print(f'Epoch {epoch+1}/{num_epochs}',
f'Train Loss: {train_loss:.4f}',
f'Val Loss: {val_loss:.4f}') #L
scheduler.step(val_loss) #M
model.train() #NUse the training listing to inspect how a PyTorch optimizer connects gradient calculation, parameter updates, epochs, validation, and device selection. section 8.2.3





































