liveBook · Manning

Adam

As explained in Build AI Drug Discovery Pipelines

Adam is an adaptive stochastic-gradient optimization algorithm that maintains running estimates of gradient moments to update model parameters.

Adam is an adaptive stochastic-gradient optimization algorithm that updates neural-network parameters while training. In drug-discovery models, it minimizes a loss such as mean squared error between predicted and measured binding affinities. section 11.5.6 section 9.2.3

Use Adam inside the minibatch training loop: compute predictions, evaluate the loss, backpropagate gradients, and call the optimizer to update parameters. The learning rate controls the update scale and can be tuned when convergence is too slow or unstable. section 8.2.3 section 9.2.3

A practical Adam configuration for affinity prediction includes:

  • Pair Adam with mean squared error when regression errors should be penalized more strongly as they become larger. section 11.5.6
  • Start with a learning rate of 0.001; adjust it when the training behavior or convergence warrants tuning. section 11.5.6 section 9.2.3
  • Combine optimizer choice with dropout, validation monitoring, cross-validation, checkpointing, and early stopping to address overfitting. section 11.5.6 section 8.2.3

Training a docking surrogate

A surrogate model learns to predict binding affinity from compound data. Its training function constructs an Adam optimizer with `torch.optim.Adam(model.parameters(), lr=0.001)`, processes shuffled batches of 32 examples, and records loss for monitoring. section 9.2.3

Listing 9.6 Training the Deep Learning Surrogate Model
def train_deep_model(
    model, X_train, y_train, device, writer=None, global_step=0, 
    epochs=10, batch_size=32, lr=0.001, **kwargs
):
    history = {"train_loss": []}
    
    criterion = nn.MSELoss()    #A
    optimizer = torch.optim.Adam(model.parameters(), lr=lr)
    
    X_train_tensor = torch.tensor(
        X_train, dtype=torch.float32
    ).to(device)    #B
    y_train_tensor = torch.tensor(
        y_train, dtype=torch.float32
    ).view(-1, 1).to(device)
    
    train_data = torch.utils.data.TensorDataset(
        X_train_tensor, y_train_tensor
    )    #C
    train_loader = torch.utils.data.DataLoader(
        train_data, batch_size=batch_size, shuffle=True
    )
    
    for epoch in range(epochs):
        model.train()
        epoch_loss = 0.0
        num_batches = 0
        
        for inputs, labels in train_loader:
            optimizer.zero_grad()    #D
            outputs = model(inputs)    #E
            loss = criterion(outputs, labels)    #F
            loss.backward()    #G
            optimizer.step()
            epoch_loss += loss.item()    #H
            num_batches += 1
        
        avg_epoch_loss = epoch_loss / num_batches
        history["train_loss"].append(avg_epoch_loss)
        if writer is not None:    #I
            writer.add_scalar(
                'Training/Loss', avg_epoch_loss,
                global_step*epochs + epoch
            )
            
        print(f"Epoch {epoch + 1}/{epochs}, Loss: {avg_epoch_loss:.4f}")
    return history

Inspect the complete training pattern, including MSE loss, Adam construction, tensor preparation, minibatch loading, and epoch iteration. section 9.2.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains Adam’s training-loop role and configuration, but does not derive its moment estimates or update equations.
Examples
Demonstrates Adam in PyTorch code for neural-network training, graph-based affinity prediction, and active-learning surrogate modeling.
What it assumes
Assumes neural-network training, loss functions, gradients, and backpropagation while supplying their operational sequence.
What you can do after
Apply it via PyTorch to train affinity-prediction models with minibatches and a tunable learning rate.
How it connects
Connects Adam with backpropagation, mean squared error, minibatch training, dropout, learning-rate scheduling, and early stopping.

Also explained in

Compare how 10 books explain adam →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.