liveBook · Manning

Mean squared error

As explained in Build AI Drug Discovery Pipelines

MSE is mean squared error, the average of the squared differences between predicted and observed numerical values.

Mean squared error (MSE) measures the error in numerical predictions. For each prediction, subtract the observed value and square the difference; then average those squared errors. Lower MSE means a better fit, and squaring gives larger errors more weight. section 4.1.4 section 4.2.1

MSE=1n∑j=1n(yj-y^j)2

The formula averages squared differences between observed values y and predictions ŷ across n data points. section 4.1.4

MSE can be both a training loss and an evaluation metric. For solubility prediction, linear regression selects weights to minimize training MSE. For a neural network predicting drug-target activity, gradients of the MSE loss guide weight updates. section 4.2.1 section 8.2

Training a PyTorch model

Use nn.MSELoss() as the criterion. For each batch, calculate predictions, compute the loss against the targets, call loss.backward(), and update the parameters with the optimizer. Track validation loss alongside training loss to guide training choices. section 8.1.2 section 8.2.3

Listing 8.8 Evaluate the Model
def evaluate_model(model, test_loader, criterion, device):
    model.eval()
    total_loss = 0.0
    predictions = []
    actuals = []
    with torch.no_grad():
        for inputs, targets in test_loader:
            inputs, targets = inputs.to(device), targets.to(device)
            outputs = model(inputs)
            loss = criterion(outputs, targets)
            total_loss += loss.item()
            predictions.extend(outputs.cpu().numpy())
            actuals.extend(targets.cpu().numpy())
    
    predictions = np.array(predictions).flatten()
    actuals = np.array(actuals).flatten()
    
    mse = mean_squared_error(actuals, predictions)
    mae = mean_absolute_error(actuals, predictions)
    
    print(f'Test Loss: {total_loss/len(test_loader):.4f}')
    >>> Test Loss: 0.9235
    print(f'Mean Squared Error: {mse:.4f}')
    >>> Mean Squared Error: 0.9249
    print(f'Mean Absolute Error: {mae:.4f}')
    >>> Mean Absolute Error: 0.7274

This evaluation listing collects test predictions and actual values, then calculates MSE and MAE with scikit-learn. section 8.2.3

MSE is in squared target units. Taking its square root gives root mean squared error (RMSE), in the original target units; mean absolute error (MAE) is another useful summary that does not square residuals. For a solubility model, an MSE of 2.29 corresponds to an RMSE of 1.51 log-molarity units. section 4.1.4

Interpreting affinity predictions

For binding affinity in pKd units, an MSE of about 0.2 gives an RMSE of about 0.45 pKd units, corresponding to roughly a 2.8-fold difference in dissociation constant. Interpret the score in its dataset’s units: KIBA scores are not pKd values. section 11.6.2

A low MSE alone does not establish that a model is ready for screening. Compare results with other metrics and consider the prediction task: ordinary MSE weights errors equally across the affinity range, even when errors among high-affinity binders matter more. A differently weighted loss may better reflect that priority. section 8.1.2 section 11.5.6

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Defines squared residuals and their average, then applies MSE in optimization and test evaluation while discussing its limitations.
Examples
Illustrates MSE with solubility regression, EGFR activity prediction, and drug-target affinity results on DAVIS and KIBA.
What you can do after
Applies MSE via PyTorch and scikit-learn to train and evaluate regression models.
How it connects
Connects MSE to linear regression, neural networks, mean absolute error, root mean squared error, and Pearson correlation.

Also explained in

Compare how 17 books explain mean squared error →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.