liveBook · Manning

Forward pass

As explained in Build AI Drug Discovery Pipelines

A forward pass is the computation that transforms model inputs into predictions by propagating them through the model.

A forward pass transforms inputs into predictions by carrying them through a model’s computations. During training, the predictions are compared with targets to calculate a loss: a measure of prediction error. section 8.2.1 section 8.2.3

Logistic regression

For inputs [1, 2, 3], weights [0.1, 0.2, 0.3], and bias 0.1, the weighted sum is 1.5. Applying the sigmoid function gives a prediction of approximately 0.82, which can be compared with the target 1 to calculate loss. section 8.2.1

Figure 8.7 Computational graph for logistic regression. The input node represents the input data fed into the graph. Learnable parameter nodes for the weights and bias are used in conjunction with the input data for intermediate computations, ending with a prediction, 𝑎, that is compared to the true class label, 𝑦, to compute the loss.

Trace the inputs, weights, and bias through intermediate computations to a prediction and loss. section 8.2.1

In a PyTorch training loop, model(inputs) performs the forward pass for a batch. After calculating loss, loss.backward() computes gradients, and optimizer.step() uses them to update parameters. These updates aim to improve predictions on later forward passes. section 8.2.3 section 8.2.1

A forward pass also produces predictions during validation. Use model.eval() to turn off training-specific behavior and torch.no_grad() to disable gradient calculation when gradients are not needed. section 8.2.3

Listing 8.7 Train the Model
def train_model(
    model, train_loader, val_loader, criterion, optimizer, 
    scheduler, device, num_epochs=10
):
    model.train()    #A
    train_losses = []    #B
    val_losses = []
    
    for epoch in range(num_epochs):    #C
        running_loss = 0.0    #D
        for inputs, targets in train_loader:
            inputs, targets = inputs.to(device), targets.to(device)
            
            optimizer.zero_grad()    #E
            outputs = model(inputs)    #F
            loss = criterion(outputs, targets)    #G
            loss.backward()    #H
            optimizer.step()    #I
            
            running_loss += loss.item()
        
        train_loss = running_loss / len(train_loader)
        train_losses.append(train_loss)
        
        model.eval()    #J
        val_loss = 0.0
        with torch.no_grad():    #K
            for inputs, targets in val_loader:
                inputs, targets = inputs.to(device), targets.to(device)
                outputs = model(inputs)
                loss = criterion(outputs, targets)
                val_loss += loss.item()
        
        val_loss /= len(val_loader)
        val_losses.append(val_loss)
        
        print(f'Epoch {epoch+1}/{num_epochs}', 
              f'Train Loss: {train_loss:.4f}',
              f'Val Loss: {val_loss:.4f}')    #L
        
        scheduler.step(val_loss)    #M
        model.train()    #N

See the forward pass in a training loop alongside gradient reset, loss calculation, backpropagation, and parameter updates. section 8.2.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Traces computations through a logistic regression graph and places forward passes within a batch training loop.
Examples
Illustrates a weighted input vector with a sigmoid prediction and a target, then uses batches in a PyTorch training loop.
What you can do after
Apply it via PyTorch using model(inputs) in training and evaluation loops.
How it connects
Connects computational graphs, loss functions, backpropagation, gradients, and parameter updates.

Also explained in

Compare how 14 books explain forward pass →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.