liveBook · Manning

Linear regression

As explained in Build AI Drug Discovery Pipelines

Linear regression is a regression method that models a linear relationship between input features and a target variable.

Linear regression predicts a continuous numerical value from input features. In drug discovery, it can estimate a molecule’s solubility from molecular descriptors: numerical properties that describe the molecule. Solubility matters because poor solubility can reduce absorption and complicate dosing. section 4.1 section 4.0

With one descriptor, simple linear regression predicts ŷ = w0 + w1x. Here x is the descriptor, w1 is its weight, and w0 is the intercept. Multiple linear regression extends this idea to several descriptors, assigning a weight to each. The prediction is a weighted sum of the features plus the intercept. section 4.1.4 section 4.2.1

Training adjusts the weights to reduce prediction errors. A residual is the difference between a measured value and its prediction. Mean squared error (MSE) averages squared residuals, giving larger errors greater emphasis. Weights can be found using a closed-form solution or iterative gradient descent. section 4.1.4 section 4.2.1 section 4.5

Predict solubility from logP

Start with logP as a single molecular descriptor and fit a line to measured solubility values. In Scikit-Learn, create a LinearRegression model, call fit with the training features and targets, and call predict to obtain estimates. The resulting line makes it possible to see how closely predictions follow the training measurements. section 4.1.4

Listing 4.4 Fit Linear Regressor to Data Set
from sklearn.linear_model import LinearRegression
from sklearn.metrics import (
    mean_absolute_error, mean_squared_error, r2_score
)
lin_reg = LinearRegression()
X_train, y_train = train_df[['logP']], train_df.Y
lin_reg.fit(X_train, y_train)
y_train_preds = lin_reg.predict(X_train)
plt.scatter(X_train, y_train, edgecolor='white', s=50)
plt.plot(X_train, y_train_preds, color='black', lw=2)

The listing fits the logP model and plots its training predictions. section 4.1.4

Evaluate predictions on unseen data as well as training data. MSE and mean absolute error are better when lower; R² can be negative on test data. Adding features never decreases training R², even when they are not useful, so adjusted R² can help assess whether extra features improve the model. Experimental solubility measurements may also differ across assays, introducing noise. section 4.1.4 section 4.1

More descriptors can capture additional information but can also make a model overly specific to its training data. Regularization constrains weights to help mitigate overfitting. Nonlinear feature transformations can extend a linear model while keeping it linear in its weights. section 3.5 section 4.3

Selecting descriptors for a small QSAR dataset

For an interpretable model of ligand properties, Lasso can first shrink irrelevant descriptor coefficients to zero. Multiple linear regression models can then be compared using subsets of the remaining descriptors. With few ligands, limiting the number of descriptors helps discourage models that fit the training set too specifically. section 6.3.1

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains weighted predictions, residuals, MSE, closed-form solutions, and gradient descent variants.
Examples
Fits solubility from logP and compares descriptor subsets for a small QSAR dataset.
What you can do after
Apply it via Scikit-Learn to fit and evaluate a solubility predictor.
How it connects
Connects linear regression to regularization, nonlinear feature transformations, cross-validation, and feature selection.

Also explained in

Compare how 28 books explain linear regression →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.