Chapter 4: Solubility Deep Dive with Linear Models
The chapter introduces why drug solubility matters in drug discovery and how linear models can help predict it from compound properties.
After this chapter you will be able to
- Solubility and therapeutic success. Explain how a drug candidate’s ability to dissolve in biological fluids affects its absorption, bioavailability, efficacy, and potential success as a therapy.
- Linear models for solubility. Explain how linear models can use compound structural features and chemical properties to predict solubility and support compound prioritization or design.
- Regression versus classification. Distinguish predicting a molecule’s numeric solubility with linear regression from assigning it to a class with linear classification.
- Solubility learning problem. Describe the inputs, target, and desired model for solubility prediction, including why performance on training data is used as a proxy for performance on unseen data.
- AqSolDB Data Access. Describe how Therapeutics Data Commons (TDC) provides access to AqSolDB and its predefined data splits for the solubility example.
- Scaffold Split Rationale. Explain why scaffold splitting gives a more challenging test than random splitting for chemically related molecules, and describe what a scaffold is.
- Solubility Scale and Distribution. Describe what the AqSolDB target Y measures and interpret the range and shape of its training-set distribution.
- Extreme Value Domain Inspection. Explain why extreme solubility values warrant inspection and how chemical context helps decide whether an entry belongs in the model's intended population.
- Single Fragment Dataset Filter. Justify excluding molecules with more than one fragment rather than removing their extra fragments.
- Target Distribution Shift Check. Explain label shift and what the train-versus-validation solubility comparison shows.
- Feature and Target Correlations. Explain what descriptor–solubility scatterplots and Pearson correlations reveal, using the section’s examples.
- Feature Engineering Without Leakage. Explain how correlations can guide feature engineering and why this exploration must use only the training split.
- Applicability Domain and Covariate Shift. Explain what an applicability domain indicates and distinguish covariate shift from a changed feature–target relationship.
- Geometric Applicability Domain Boundaries. Describe how bounding boxes and convex hulls classify new molecules as in or out of the training-data region, including their limitations.
- Single Descriptor Regression Model. Explain how simple linear regression uses logP to predict solubility with a best-fitting line, and identify the roles of the feature, prediction, weights, and intercept.
- Solubility Model Performance. Explain why the section reports RMSE and what the simple model’s training result implies.
- Weighted feature predictions. Explain how a linear model combines an instance’s feature values with adjustable weights to predict solubility, including the role of the bias term.
- Mean squared error objective. Describe how prediction errors become the mean squared error cost and why fitting seeks weights that minimize it.
- OLS closed-form solution. Explain that Scikit-Learn’s LinearRegression uses ordinary least squares by default to compute weights that minimize the cost function directly, rather than adjusting them iteratively.
- Gradient Descent Weight Updates. Explain how gradient descent starts with weights, uses the cost gradient to update them, and seeks a minimum. Relate this to the convex MSE cost of linear regression.
- Feature Scaling Before Descent. Justify scaling features before gradient descent and describe why this example uses StandardScaler fitted only on training data rather than min-max scaling.
- ESOL Solubility Benchmark. Describe ESOL as a published linear-regression baseline for aqueous solubility and state the validation RMSE this chapter aims to beat.
- Fair ESOL Comparison. Explain why the section removes molecules shared with ESOL's training data from validation and notes differences in descriptor implementations.
- Y-Scrambling Robustness Check. Trace how y-scrambling creates a comparison baseline and explain what similar or degraded performance on scrambled data suggests.
- Ridge Pipeline and Missing Values. Describe the ridge regression pipeline’s preprocessing steps and explain why its median imputation is fitted on training data.
- Coefficients Versus Feature Selection. Explain what ridge coefficient magnitudes suggest about descriptor importance and distinguish coefficient shrinkage from removing features.
- Subset Selection Tradeoffs. Distinguish exhaustive best subset selection from forward and backward sequential feature selection, including their computational and optimality tradeoffs.
- Final Model and Baselines. Describe which model was selected and how its test RMSE compared with the ESOL and y-scrambled baselines.
- Loss Choice Defines Model. Explain why SGDRegressor is not automatically SVR and how the tested loss functions treat large and small errors.
- Learning Rate Loss Curves. Explain how the learning-rate schedules differ and what the loss curves showed about training stability and epoch choice.
- Validation Based Early Stopping. Describe how early stopping uses validation performance and why it was useful for the selected model.
- Bias and variance trade-off. Explain how high bias and high variance cause different kinds of prediction error, and why a model must be flexible enough to capture real patterns without fitting noise.
- High Bias and Underfitting. Explain why the 2nd-order polynomial, model B, underfits data sampled from a noisy 7th-order target function. Describe how its training and test performance relate.
- High Variance and Overfitting. Explain why the more complex 20th-order model V can have higher test error than model B despite being flexible enough to represent the target function. Describe what variance means in this example.
- Learning Curve Error Patterns. Explain how training and test errors typically change as training-set size grows, and what their gap and convergence level indicate about variance and bias.
- RANSAC and SVR Data Choices. Compare what the RANSAC and SVR learning curves imply about collecting more data, including the data-starved case.
- Validation curve trade-offs. Explain how validation curves relate hyperparameter values to model complexity, training error, and performance on unseen data, and how they help identify a useful balance.
What it builds on
- Property-based Filters (chapter 2) · Relates solubility to the use of compound properties when assessing drug candidates.
- Training our Linear Model (chapter 3) · The earlier training of a linear classification model.
- Evaluating our Model (chapter 3) · The earlier use of random splitting to validate classification models.
- Validating & Standardizing SMILES (chapter 3) · The earlier approach of standardizing molecules by removing fragments.
- Molecular Descriptors (chapter 2) · The earlier introduction to molecular descriptors.
Section by section
Introduction
The chapter introduces why drug solubility matters in drug discovery and how linear models can help predict it from compound properties.
- Solubility and therapeutic success: Explain how a drug candidate’s ability to dissolve in biological fluids affects its absorption, bioavailability, efficacy, and potential success as a therapy.
Why it matters: Solubility is worth predicting because it can affect whether a compound works well as a drug. - Bioavailability and administration routes: Distinguish the definition of bioavailability from the factors that can limit it, especially for oral drugs compared with intravenous drugs.
Why it matters: This distinction prevents a low oral bioavailability result from being mistaken for proof of a single cause. - Linear models for solubility: Explain how linear models can use compound structural features and chemical properties to predict solubility and support compound prioritization or design.
Why it matters: It connects a drug-development problem to the purpose of the chapter’s modeling work.
Key terms: solubility, bioavailability, absorption, systemic circulation, intravenous (IV), oral bioavailability, first-pass metabolism, linear models, structural features, chemical properties
4.1 Solubility with Linear Regression
This section frames solubility prediction as a linear regression problem and explains how experimental measurements and their noise affect what the model can learn.
- Regression versus classification: Distinguish predicting a molecule’s numeric solubility with linear regression from assigning it to a class with linear classification.
Why it matters: The distinction clarifies why solubility is treated as a numeric prediction rather than a class label. - Solubility learning problem: Describe the inputs, target, and desired model for solubility prediction, including why performance on training data is used as a proxy for performance on unseen data.
Why it matters: It connects the molecule data used for training to the predictions the model is meant to make in practice. - Experimental measurement uncertainty: Explain why measured solubility values may be inconsistent and what assumption this section makes about its chosen data set.
Why it matters: It keeps the reader from treating experimental labels as perfectly consistent across studies.
Key terms: linear classification, linear regression, continuous numeric value, linear decision boundary, molecule descriptors, solubility, hypothesis g, training data set, unknown distribution P(x,y), aggregate error
4.1.1 Load the Data
This section introduces AqSolDB as a source of measured solubility data and explains why the example loads scaffold-based rather than random data splits.
- AqSolDB Data Access: Describe how Therapeutics Data Commons (TDC) provides access to AqSolDB and its predefined data splits for the solubility example.
Why it matters: It identifies both the measured values used for the solubility task and the source of the data splits. - Scaffold Split Rationale: Explain why scaffold splitting gives a more challenging test than random splitting for chemically related molecules, and describe what a scaffold is.
Why it matters: The way molecules are split changes how convincing a model’s test results are.
Key terms: Therapeutics Data Commons, TDC, AqSolDB, scaffold-based data splits, random splitting, scaffold, scaffold splitting, model validation
4.1.2 Target Variable Distribution
The section examines AqSolDB solubility values to identify unusual molecules, justify a fragment-based cleaning decision, and compare training and validation target distributions.
- Solubility Scale and Distribution: Describe what the AqSolDB target Y measures and interpret the range and shape of its training-set distribution.
Why it matters: Reading the target scale correctly makes the unusually broad distribution meaningful. - Extreme Value Domain Inspection: Explain why extreme solubility values warrant inspection and how chemical context helps decide whether an entry belongs in the model's intended population.
Why it matters: Chemical context keeps data cleaning focused on the molecules the model is meant to handle. - IQR Outlier Flagging: Describe how the interquartile range method flags potential solubility outliers.
Why it matters: This gives a statistical way to flag values for further scrutiny rather than relying only on visual inspection. - Single Fragment Dataset Filter: Justify excluding molecules with more than one fragment rather than removing their extra fragments.
Why it matters: The filter avoids retaining a solubility measurement after altering the molecular composition that influenced it. - Target Distribution Shift Check: Explain label shift and what the train-versus-validation solubility comparison shows.
Why it matters: Comparing target distributions helps reveal when a model may face values unlike those it trained on.
Key terms: Y, aqueous solubility, logS, log molarity, target variable distribution, outliers, chemical domain knowledge, organic molecules, hydrogen fluoride, interquartile range
4.1.3 Feature Computation & Correlation
This section computes molecular descriptors, examines their relationships with solubility, and uses training-data boundaries to discuss where model predictions may be reliable.
- Descriptor Computation and Coverage: Describe how the section computes molecular descriptors and explain why the chosen descriptors do not capture every factor affecting aqueous solubility.
Why it matters: The model can only use molecular information represented in its features. - Feature and Target Correlations: Explain what descriptor–solubility scatterplots and Pearson correlations reveal, using the section’s examples.
Why it matters: These checks show which features appear informative and flag unusual molecules before modeling. - Feature Engineering Without Leakage: Explain how correlations can guide feature engineering and why this exploration must use only the training split.
Why it matters: Feature choices can affect learning, and making them with held-out data would compromise the evaluation. - Applicability Domain and Covariate Shift: Explain what an applicability domain indicates and distinguish covariate shift from a changed feature–target relationship.
Why it matters: A prediction deserves more caution when the new molecule differs substantially from the training molecules. - Geometric Applicability Domain Boundaries: Describe how bounding boxes and convex hulls classify new molecules as in or out of the training-data region, including their limitations.
Why it matters: A simple in-range check is useful, but it should not be mistaken for proof that a prediction is reliable.
Key terms: RDKit, descriptors, ionization state, scatterplots, Pearson’s correlation coefficient, logP, TPSA, outliers, feature engineering, multicollinearity
4.1.4 Linear Regression
The section fits a simple linear regression model to predict solubility from logP, explains how to assess its fit, and shows that its training performance is inadequate.
- Single Descriptor Regression Model: Explain how simple linear regression uses logP to predict solubility with a best-fitting line, and identify the roles of the feature, prediction, weights, and intercept.
Why it matters: This connects the molecular descriptor to the model’s solubility prediction. - Residuals and Error Measures: Describe a residual and explain how MSE and MAE summarize prediction errors, including why MSE emphasizes larger errors.
Why it matters: Error measures make the quality of a fitted line assessable rather than merely visual. - R Squared Feature Adjustment: Distinguish R² from adjusted R² when features are added to a model.
Why it matters: Adjusted R² helps avoid mistaking extra features for a genuine improvement. - Solubility Model Performance: Explain why the section reports RMSE and what the simple model’s training result implies.
Why it matters: The poor training result motivates improving the model before applying it to new compounds.
Key terms: simple linear regression, logP, solubility, weights, bias, intercept, residual, mean squared error (MSE), mean absolute error (MAE), coefficient of determination (𝑅2)
4.2 The Learning Algorithm
This introduction raises questions about how linear models learn and suggests that answering them may help explain and improve model performance.
4.2.1 Linear Models
This section defines how a linear model predicts solubility from weighted features and how mean squared error identifies a best fit.
- Weighted feature predictions: Explain how a linear model combines an instance’s feature values with adjustable weights to predict solubility, including the role of the bias term.
Why it matters: This is the prediction rule whose weights the learning algorithm must fit. - Mean squared error objective: Describe how prediction errors become the mean squared error cost and why fitting seeks weights that minimize it.
Why it matters: MSE gives the model a concrete measure of fit to minimize. - Higher-dimensional best fit: Explain how the best-fit idea extends from a regression line to multiple features, and state what the section says about finding optimal linear-regression weights.
Why it matters: It connects the pictured regression line to the model’s actual multi-feature setting without assuming that fitting must proceed by trial and error.
Key terms: linear model, feature vector, weight vector, dot product, bias term, squared error, mean squared error (MSE), cost function, weights, line of best fit
4.2.2 Ordinary Least Squares (OLS)
This section explains how ordinary least squares directly fits linear regression weights with the Normal equation, and where that approach has limitations.
- OLS closed-form solution: Explain that Scikit-Learn’s LinearRegression uses ordinary least squares by default to compute weights that minimize the cost function directly, rather than adjusting them iteratively.
Why it matters: It explains what the default linear regression fitting method does and how it differs from iterative optimization. - Training data matrix notation: Describe what X, y, and the resulting weights represent in the section’s account of the Normal equation.
Why it matters: The notation connects the Normal equation to the training examples and predictions it fits. - OLS solution limitations: Explain what happens when XᵀX is not invertible and why the Normal equation can become slow as the feature count grows.
Why it matters: A direct solution is convenient, but it does not guarantee unique weights or fast computation.
Key terms: ordinary least squares (OLS), closed-form solution, Normal equation, cost function, Matrix X, vector y, bias, weights, pseudo-inverse, not invertible
4.2.3 Gradient Descent
This section explains how gradient descent updates model weights, what affects its convergence, and why feature scaling matters before gradient-based training.
- Gradient Descent Weight Updates: Explain how gradient descent starts with weights, uses the cost gradient to update them, and seeks a minimum. Relate this to the convex MSE cost of linear regression.
Why it matters: This is the basic weight-learning routine behind the gradient-based models discussed here and later. - Step Size and Stopping: Describe how the learning rate and gradient affect update size, and state the combination of stopping criteria recommended in the section.
Why it matters: Step size and stopping rules affect whether training makes progress and when it ends. - Stochastic Versus Batch Updates: Distinguish stochastic and mini-batch gradient descent from full-batch updates, including their speed and convergence tradeoff.
Why it matters: The choice of how much data to use per update trades computational efficiency against steadiness near the minimum. - Gradient Cliffs and Clipping: Explain how a gradient cliff can disrupt convergence and how gradient clipping addresses it.
Why it matters: Clipping is a way to keep unusually large gradients from causing an excessive update. - Feature Scaling Before Descent: Justify scaling features before gradient descent and describe why this example uses StandardScaler fitted only on training data rather than min-max scaling.
Why it matters: Comparable feature scales help gradient descent approach the minimum without one feature’s scale dominating its direction.
Key terms: gradient descent, gradient, negative gradient, MSE, convex, global minimum, learning rate, gradient norm, termination criteria, marginal decreases in cost
4.3 Touring Scikit-Learn Linear Models
This section previews comparing linear regression models using the full feature space, starting with Ridge regression because some features may be redundant or irrelevant.
4.3.1 Defining a Benchmark
This section defines an ESOL benchmark for solubility prediction and explains how y-scrambling can test whether a model has learned meaningful relationships.
- ESOL Solubility Benchmark: Describe ESOL as a published linear-regression baseline for aqueous solubility and state the validation RMSE this chapter aims to beat.
Why it matters: A published reference point makes the performance of new solubility models interpretable. - Fair ESOL Comparison: Explain why the section removes molecules shared with ESOL's training data from validation and notes differences in descriptor implementations.
Why it matters: Controlling overlap and descriptor differences helps make the benchmark comparison meaningful. - Y-Scrambling Robustness Check: Trace how y-scrambling creates a comparison baseline and explain what similar or degraded performance on scrambled data suggests.
Why it matters: A model that performs just as well after its targets are scrambled may be exploiting chance patterns rather than predictive chemistry.
Key terms: Estimated Solubility (ESOL), aqueous solubility, cLogP, MW, RB, AP, InChI keys, validation data set, overlapping molecules, RDKit
4.3.2 Ridge Regression & Feature Selection
The section fits a ridge regression pipeline for solubility prediction, examines its coefficients, and tests whether sequential feature selection improves it.
- Ridge Pipeline and Missing Values: Describe the ridge regression pipeline’s preprocessing steps and explain why its median imputation is fitted on training data.
Why it matters: A descriptor pipeline must still handle molecules whose computed features contain missing values. - Ridge Search and Scoring: Explain what the randomized search varied, what the best validation result was, and why its scoring function used negative MSE.
Why it matters: The search result shows what this ridge model achieved before attempting feature selection. - Coefficients Versus Feature Selection: Explain what ridge coefficient magnitudes suggest about descriptor importance and distinguish coefficient shrinkage from removing features.
Why it matters: A small ridge coefficient is not evidence that a descriptor has been removed from the model. - Subset Selection Tradeoffs: Distinguish exhaustive best subset selection from forward and backward sequential feature selection, including their computational and optimality tradeoffs.
Why it matters: The tradeoff explains why the section tries a greedy selector rather than testing every descriptor combination. - Selected Features and Performance: Describe the subset selected in the section’s ridge experiment and what happened to training and validation errors.
Why it matters: Selecting fewer descriptors did not improve this model’s measured performance.
Key terms: Ridge, L2 regularization, SimpleImputer, PolynomialFeatures, StandardScaler, median, randomized search, alpha, polynomial feature degree, RMSE
4.3.3 Building a More Performant Model
The section selects and evaluates an SGD-trained, support-vector-style solubility model, then explains how its loss function, learning-rate schedule, and early stopping affect training.
- Final Model and Baselines: Describe which model was selected and how its test RMSE compared with the ESOL and y-scrambled baselines.
Why it matters: Comparing against baselines shows whether the selected model improves on meaningful reference points. - Loss Choice Defines Model: Explain why SGDRegressor is not automatically SVR and how the tested loss functions treat large and small errors.
Why it matters: The loss setting determines both what kind of fit SGDRegressor produces and how it responds to noisy solubility measurements. - Learning Rate Loss Curves: Explain how the learning-rate schedules differ and what the loss curves showed about training stability and epoch choice.
Why it matters: The curves show why step size and training duration can change validation performance even for the same model. - Validation Based Early Stopping: Describe how early stopping uses validation performance and why it was useful for the selected model.
Why it matters: Stopping based on validation performance can preserve a better model while avoiding needless epochs.
Key terms: SVR, RMSE, ESOL baseline, y-scrambled baseline, SGDRegressor, squared error loss, Huber loss, epsilon-insensitive loss, PolynomialFeatures, loss curve
4.4 Bias-Variance Decomposition
This section explains how bias and variance account for underfitting and overfitting, and why model complexity must balance the two.
- Bias and variance trade-off: Explain how high bias and high variance cause different kinds of prediction error, and why a model must be flexible enough to capture real patterns without fitting noise.
Why it matters: This trade-off helps explain why making a model more complicated does not always improve its predictions.
Key terms: bias, variance, underfits, overfits
4.4.1 A Case Study in Polynomials
A noisy polynomial example shows how high bias and high variance produce different failures, then introduces curves for diagnosing them.
- High Bias and Underfitting: Explain why the 2nd-order polynomial, model B, underfits data sampled from a noisy 7th-order target function. Describe how its training and test performance relate.
Why it matters: It helps you recognize when a model fails because it cannot capture the underlying pattern. - High Variance and Overfitting: Explain why the more complex 20th-order model V can have higher test error than model B despite being flexible enough to represent the target function. Describe what variance means in this example.
Why it matters: Greater flexibility does not guarantee better predictions on unseen data. - Contextual Model Choice: Describe the circumstances the section gives for accepting a high-bias or high-variance model.
Why it matters: Model selection depends on what kind of performance the application needs. - Curve-Based Model Diagnosis: Describe the basic performance comparison used to construct learning and validation curves in this section.
Why it matters: Comparing training and test performance makes the model's behavior easier to diagnose.
Key terms: model B, high bias, underfits, target function, model V, low bias, high variance, overfits, noise, safety
4.4.2 Learning Curves
This section uses learning curves to diagnose bias and variance and decide whether more training data or a different model is likely to help.
- Learning Curve Error Patterns: Explain how training and test errors typically change as training-set size grows, and what their gap and convergence level indicate about variance and bias.
Why it matters: The shape of a learning curve helps identify why a model is underperforming. - RANSAC and SVR Data Choices: Compare what the RANSAC and SVR learning curves imply about collecting more data, including the data-starved case.
Why it matters: These curves show why the best next step depends on the model and how much data is available.
Key terms: learning curves, train error, test error, bias, variance, RANSAC, SVR, train MSE, test MSE, overfitting
4.4.3 Validation Curves
This section explains how validation curves reveal the effect of hyperparameters on model complexity and generalization, using an SVR estimator as an example.
- Validation curve trade-offs: Explain how validation curves relate hyperparameter values to model complexity, training error, and performance on unseen data, and how they help identify a useful balance.
Why it matters: A validation curve helps you choose a hyperparameter for performance on unseen data rather than training fit alone. - SVR hyperparameter curve patterns: Describe what the section’s SVR validation curves show for L2 regularization, polynomial feature degree, and epsilon-tube width.
Why it matters: The SVR curves show why different hyperparameters must be interpreted by their observed effects, not by assuming that more complexity always helps.
Key terms: validation curves, hyperparameters, model complexity, high bias, high variance, underfitting, SVR, L2 regularization, alpha, polynomial feature degree
4.5 Summary
Where this is used later
- Training & Applying our Neural Network (chapter 8) · Earlier regression modeling provides context for predicting continuous values.
- Binary Classification of CYP3A4 Inhibition (chapter 5) · Introduces linear models, the model family revisited here for classification.
- Logistic Regression in Theory (chapter 5) · The linear model’s weighted feature signal.
- The XGBoost Magic Trick (chapter 6) · The earlier introduction to gradient-based optimization.
- Models as Computational Graphs (chapter 8) · Introduces gradient-based parameter changes.
- Decision Tree Bias-Variance Trade Offs (chapter 5) · Assumes the distinction between bias and variance.
- Implementing a Neural Network (chapter 8) · The distinction between fitting training data and generalizing.
Concepts explained in this chapter
- linear regressionLinear regression is a regression method that models a linear relationship between input features and a target variable.
- support vector regressionSVR is support vector regression, a supervised regression method that fits a function using support vectors and an epsilon-insensitive loss.
- stochastic gradient descentMini-batch gradient descent is an optimization method that computes each parameter update from a small subset of the training data.
- learning rateA learning rate is an optimization step-size configuration that controls the magnitude of parameter updates during iterative learning.
- learning curveA learning curve is a diagnostic plot showing model performance or error as a function of training-set size.
- biasStatistical bias is systematic error caused by a model's simplifying assumptions that prevent it from capturing the true structure of the data.
- model complexityModel complexity is the degree of flexibility or representational richness of a model's function class.
- varianceDistribution variance is the expected squared deviation from a distribution's mean and measures its spread.
- RANSACRANSAC is a robust estimation technique that fits a model using consensus subsets of data while limiting the influence of outliers.
- molecular descriptorA molecular descriptor is a numerical or symbolic representation of a molecule's physicochemical, structural, topological, or quantum-mechanical properties.
- AqSolDBAqSolDB is a curated database of aqueous solubility measurements for chemical compounds and molecules.
- root mean squared errorRoot mean squared error (RMSE) is the square root of the mean squared differences between predicted and observed values, expressed in the target's original units.
- objective functionA cost function measures aggregate prediction error across multiple data instances, with lower values representing better performance.
- validation curveA validation curve is a plot of model performance as a selected model or hyperparameter dimension changes.
- bounding box methodThe bounding box method defines a region by the minimum and maximum value of each feature or descriptor dimension.
- convex hullA convex hull is the smallest convex region containing a specified set of points.
- correlationThe Pearson correlation coefficient is a statistic ranging from -1 to 1 that measures the strength and direction of a linear association between two continuous variables.
- covariate shiftCovariate shift is a distribution shift in which the input-feature distribution changes between training and deployment while the conditional target relationship remains unchanged.
- lipophilicityLogP is the logarithm of a compound's octanol-water partition coefficient and is a measure of lipophilicity.
- topological polar surface areaTopological polar surface area (TPSA) is a molecular descriptor estimating the surface area contributed by polar atoms and their bonding contributions.
- feature scalingFeature scaling transforms feature values to comparable numerical ranges before model fitting.
- gradient clippingGradient clipping is a training technique that limits gradient values or their norm to prevent excessively large parameter updates.
- gradient descentGradient descent is an iterative optimization method that updates parameters in the direction that decreases a differentiable objective function.
- interceptAn intercept is the constant term in a linear model that represents the predicted output when all input features are zero.
- mean squared errorMSE is mean squared error, the average of the squared differences between predicted and observed numerical values.
- anomalyAn outlier is a data point whose value is unusually distant from the pattern or distribution of the other observations.
- SGDRegressorSGDRegressor is a Scikit-Learn estimator that fits linear models by minimizing a regularized empirical loss with stochastic gradient descent.
- early stoppingEarly stopping is a training-control technique that terminates iterative model fitting when additional updates no longer improve a selected evaluation criterion.
- epsilon-insensitive lossEpsilon-insensitive loss assigns zero penalty to errors within an epsilon margin around the target and increases linearly for errors outside that margin.
- bias-variance tradeoffThe bias-variance trade-off is the balance between systematic error from an overly simple model and variability from an overly flexible model.
- exploding gradientThe exploding gradient problem is a training failure in which gradients grow excessively during backpropagation, causing unstable parameter updates.
- gradientA gradient is a vector containing the partial derivatives of a scalar function with respect to its variables and pointing toward steepest increase.
- gradient cliffA gradient cliff is a region of an objective landscape where the cost changes steeply over a small parameter displacement.
- ridge regressionRidge is a linear-model regularization method that applies an L2 penalty to coefficient weights.
- interquartile rangeThe interquartile range is the difference between the third and first quartiles of a distribution.
- label shiftLabel shift is a distribution shift in which the distribution of target values differs between training data and evaluation or application data.
- logSlogS is the base-ten logarithm of a compound's molar solubility, with each unit representing a tenfold change in solubility.
- molecular fragmentA molecular fragment is a small chemical structure that can bind a biological target and serve as a starting point or building block for a larger compound.
- solubilitySolubility is the amount of a substance that can dissolve in a specified solvent under specified conditions.
- generalizationGeneralization is a model's ability to maintain predictive performance on previously unseen data drawn from the relevant distribution.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
