liveBook · Manning

Chapter 4: Solubility Deep Dive with Linear Models

What you will learn in chapter 4 of Build AI Drug Discovery Pipelines

The chapter introduces why drug solubility matters in drug discovery and how linear models can help predict it from compound properties.

After this chapter you will be able to

What it builds on

Section by section

Introduction

The chapter introduces why drug solubility matters in drug discovery and how linear models can help predict it from compound properties.

Key terms: solubility, bioavailability, absorption, systemic circulation, intravenous (IV), oral bioavailability, first-pass metabolism, linear models, structural features, chemical properties

4.1 Solubility with Linear Regression

This section frames solubility prediction as a linear regression problem and explains how experimental measurements and their noise affect what the model can learn.

Key terms: linear classification, linear regression, continuous numeric value, linear decision boundary, molecule descriptors, solubility, hypothesis g, training data set, unknown distribution P(x,y), aggregate error

4.1.1 Load the Data

This section introduces AqSolDB as a source of measured solubility data and explains why the example loads scaffold-based rather than random data splits.

Key terms: Therapeutics Data Commons, TDC, AqSolDB, scaffold-based data splits, random splitting, scaffold, scaffold splitting, model validation

4.1.2 Target Variable Distribution

The section examines AqSolDB solubility values to identify unusual molecules, justify a fragment-based cleaning decision, and compare training and validation target distributions.

Key terms: Y, aqueous solubility, logS, log molarity, target variable distribution, outliers, chemical domain knowledge, organic molecules, hydrogen fluoride, interquartile range

4.1.3 Feature Computation & Correlation

This section computes molecular descriptors, examines their relationships with solubility, and uses training-data boundaries to discuss where model predictions may be reliable.

Key terms: RDKit, descriptors, ionization state, scatterplots, Pearson’s correlation coefficient, logP, TPSA, outliers, feature engineering, multicollinearity

4.1.4 Linear Regression

The section fits a simple linear regression model to predict solubility from logP, explains how to assess its fit, and shows that its training performance is inadequate.

Key terms: simple linear regression, logP, solubility, weights, bias, intercept, residual, mean squared error (MSE), mean absolute error (MAE), coefficient of determination (𝑅2)

4.2 The Learning Algorithm

This introduction raises questions about how linear models learn and suggests that answering them may help explain and improve model performance.

4.2.1 Linear Models

This section defines how a linear model predicts solubility from weighted features and how mean squared error identifies a best fit.

Key terms: linear model, feature vector, weight vector, dot product, bias term, squared error, mean squared error (MSE), cost function, weights, line of best fit

4.2.2 Ordinary Least Squares (OLS)

This section explains how ordinary least squares directly fits linear regression weights with the Normal equation, and where that approach has limitations.

Key terms: ordinary least squares (OLS), closed-form solution, Normal equation, cost function, Matrix X, vector y, bias, weights, pseudo-inverse, not invertible

4.2.3 Gradient Descent

This section explains how gradient descent updates model weights, what affects its convergence, and why feature scaling matters before gradient-based training.

Key terms: gradient descent, gradient, negative gradient, MSE, convex, global minimum, learning rate, gradient norm, termination criteria, marginal decreases in cost

4.3 Touring Scikit-Learn Linear Models

This section previews comparing linear regression models using the full feature space, starting with Ridge regression because some features may be redundant or irrelevant.

4.3.1 Defining a Benchmark

This section defines an ESOL benchmark for solubility prediction and explains how y-scrambling can test whether a model has learned meaningful relationships.

Key terms: Estimated Solubility (ESOL), aqueous solubility, cLogP, MW, RB, AP, InChI keys, validation data set, overlapping molecules, RDKit

4.3.2 Ridge Regression & Feature Selection

The section fits a ridge regression pipeline for solubility prediction, examines its coefficients, and tests whether sequential feature selection improves it.

Key terms: Ridge, L2 regularization, SimpleImputer, PolynomialFeatures, StandardScaler, median, randomized search, alpha, polynomial feature degree, RMSE

4.3.3 Building a More Performant Model

The section selects and evaluates an SGD-trained, support-vector-style solubility model, then explains how its loss function, learning-rate schedule, and early stopping affect training.

Key terms: SVR, RMSE, ESOL baseline, y-scrambled baseline, SGDRegressor, squared error loss, Huber loss, epsilon-insensitive loss, PolynomialFeatures, loss curve

4.4 Bias-Variance Decomposition

This section explains how bias and variance account for underfitting and overfitting, and why model complexity must balance the two.

Key terms: bias, variance, underfits, overfits

4.4.1 A Case Study in Polynomials

A noisy polynomial example shows how high bias and high variance produce different failures, then introduces curves for diagnosing them.

Key terms: model B, high bias, underfits, target function, model V, low bias, high variance, overfits, noise, safety

4.4.2 Learning Curves

This section uses learning curves to diagnose bias and variance and decide whether more training data or a different model is likely to help.

Key terms: learning curves, train error, test error, bias, variance, RANSAC, SVR, train MSE, test MSE, overfitting

4.4.3 Validation Curves

This section explains how validation curves reveal the effect of hyperparameters on model complexity and generalization, using an SVR estimator as an example.

Key terms: validation curves, hyperparameters, model complexity, high bias, high variance, underfitting, SVR, L2 regularization, alpha, polynomial feature degree

4.5 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 3: Ligand-based Screening: Machine Learning Chapter 5: Classification: Cytochrome P450 Inhibition →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.