liveBook · Manning

Ridge regression

As explained in Build AI Drug Discovery Pipelines

Ridge is a linear-model regularization method that applies an L2 penalty to coefficient weights.

Ridge regression is a linear regressor that adds an L2 penalty, shrinking coefficient weights toward zero rather than usually removing features. In molecular solubility prediction, this helps control redundant or weak descriptors while retaining the full feature space. section 4.3 section 4.3.2

A practical ridge workflow:

  • Impute missing descriptor values using training-set medians, then apply polynomial expansion and standard scaling inside a pipeline. section 4.3.2
  • Search regularization strength alpha and polynomial degree with randomized cross-validation, evaluating validation RMSE. section 4.3.2
  • Inspect coefficient magnitudes as influence indicators, but do not treat them as exact feature selection. section 4.3.2

Solubility prediction

A tuned ridge model using molecular descriptors achieved a validation RMSE of 1.489 and selected polynomial degree 1, indicating that nonlinear feature expansion did not improve this result. section 4.3.2

Listing 4.7 Ridge Regressor Randomized Search
from scipy.stats import loguniform
from sklearn.impute import SimpleImputer
from sklearn.linear_model import Ridge
from sklearn.model_selection import RandomizedSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
 
features = pd.Index(RDKIT_DESCRIPTORS.keys())    #A
X_train, y_train = train_df[features.tolist()], train_df.Y
X_val, y_val = valid_df[features.tolist()], valid_df.Y
 
param_dist = {
    'ridge__alpha' : loguniform(1e-6, 1e2),    #B
    'polynomialfeatures__degree' : [1, 2],
}
 
pipe = Pipeline([
  ('imputer', SimpleImputer(strategy="median")),
  ('polynomialfeatures', PolynomialFeatures()),
  ('standardscaler', StandardScaler(with_mean=True, with_std=True)),
  ('ridge', Ridge()),
])
 
rs_estimator = RandomizedSearchCV(
  estimator=pipe,
  param_distributions=param_dist,
  scoring="neg_mean_squared_error",
  cv=5,
  n_iter=20,
)
 
rs_estimator.fit(X_train, y_train)
best_ridge_estimator = rs_estimator.best_estimator_
y_train_preds = best_ridge_estimator.predict(X_train)
y_val_preds = best_ridge_estimator.predict(X_val)
   print(rs_estimator.best_params_)    #C
 
train_rmse = np.sqrt(mean_squared_error(y_train, y_train_preds))
val_rmse = np.sqrt(mean_squared_error(y_val, y_val_preds))
print(f"Train RMSE: {train_rmse:.4f}\nValidation RMSE: {val_rmse:.4f}")
# Output -> Train RMSE: 1.3687, Validation RMSE: 1.4890

Use the runnable pipeline and randomized-search listing to configure imputation, feature transformations, scaling, ridge regression, and alpha tuning. section 4.3.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains L2 shrinkage, preprocessing, hyperparameter search, validation RMSE, and coefficient interpretation, but not the optimization derivation.
Examples
Demonstrates molecular solubility prediction, descriptor coefficients, randomized alpha search, and a 1.489 validation RMSE.
What it assumes
Assumes molecular descriptors and train-validation data, while demonstrating missing-value handling and regularized regression practice.
What you can do after
Operate/configure a Scikit-Learn ridge pipeline and evaluate it with validation RMSE and coefficient inspection.
How it connects
Connects linear regression with L2 regularization, molecular descriptors, imputation, scaling, polynomial features, and feature selection.

Also explained in

Compare how 4 books explain ridge regression →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.