liveBook · Manning

Training dataset

As explained in Build AI Drug Discovery Pipelines

A training set is the collection of examples provided to a learning algorithm for fitting a model.

A training dataset, or training set, is the collection of examples used to fit a model. Each example is a training instance or sample. For drug discovery, examples can pair molecule descriptors with experimentally measured solubility, so a model can learn to predict solubility from those descriptors. section 1.2.2 section 4.1

Training data helps a model learn patterns rather than simply memorize examples. A separate test set checks how well those patterns generalize to unseen inputs. If a dog-recognition training set contains only German Shepherds, for example, a model may fail to recognize other breeds. section 1.2.2

In supervised neural-network training, each batch supplies inputs and corresponding targets. The model predicts outputs, a loss function measures their difference from the targets, and an optimizer updates the model’s parameters. An epoch is one pass through the training dataset. Validation data provides a separate loss to monitor during training. section 8.2.3

Listing 8.7 Train the Model
def train_model(
    model, train_loader, val_loader, criterion, optimizer, 
    scheduler, device, num_epochs=10
):
    model.train()    #A
    train_losses = []    #B
    val_losses = []
    
    for epoch in range(num_epochs):    #C
        running_loss = 0.0    #D
        for inputs, targets in train_loader:
            inputs, targets = inputs.to(device), targets.to(device)
            
            optimizer.zero_grad()    #E
            outputs = model(inputs)    #F
            loss = criterion(outputs, targets)    #G
            loss.backward()    #H
            optimizer.step()    #I
            
            running_loss += loss.item()
        
        train_loss = running_loss / len(train_loader)
        train_losses.append(train_loss)
        
        model.eval()    #J
        val_loss = 0.0
        with torch.no_grad():    #K
            for inputs, targets in val_loader:
                inputs, targets = inputs.to(device), targets.to(device)
                outputs = model(inputs)
                loss = criterion(outputs, targets)
                val_loss += loss.item()
        
        val_loss /= len(val_loader)
        val_losses.append(val_loss)
        
        print(f'Epoch {epoch+1}/{num_epochs}', 
              f'Train Loss: {train_loss:.4f}',
              f'Val Loss: {val_loss:.4f}')    #L
        
        scheduler.step(val_loss)    #M
        model.train()    #N

Follow the training loop to see how batches of training inputs and targets drive loss calculation and parameter updates. section 8.2.3

Solubility measurements

A solubility training set pairs molecule descriptors with assay measurements. Measurements can differ across studies or experimental settings, and this noise can affect how well the trained model performs in production. section 4.1

Compare training and validation loss when deciding how long to train. In one run, training loss falls to 0.2256 by epoch 50, while validation loss is 0.6720 and has largely saturated. This can motivate fewer epochs or early stopping; a separate test set is still needed to evaluate unseen data. section 8.2.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains training samples, batches, epochs, loss-based parameter updates, and validation monitoring, with an implementation of the training loop.
Examples
Uses dog photos, molecule descriptors paired with solubility measurements, and a neural-network training loop.
What you can do after
Apply it via PyTorch to train a neural network in batches and monitor validation loss.
How it connects
Connects training datasets to generalization, overfitting, validation sets, test sets, and measurement noise.

Also explained in

Compare how 50 books explain training dataset →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.