Chapter 8: Introduction to Deep Learning
This introduction previews the chapter’s move from earlier ligand-based machine learning work to applied deep learning with PyTorch and outlines the sections ahead.
After this chapter you will be able to
- Deep learning modeling promise. Explain that neural networks can learn complex patterns from large amounts of data, making deep learning promising for protein–ligand interaction modeling where traditional computational methods have faced challenges.
- Kinase signaling and opportunity. Explain how protein kinases alter signaling and why they remain important drug targets despite existing approved inhibitors.
- EGFR cancer target rationale. Explain why EGFR is the chapter’s kinase target and what EGFR-targeting drugs aim to interrupt.
- EGFR activity data interpretation. Describe what EGFR pIC50 represents, how it relates to potency, and what care the section calls for when assembling activity data.
- Hinge binders and selectivity. Explain why the Enamine hinge binder library is a focused source of possible EGFR inhibitors and why hinge binding alone does not establish EGFR selectivity.
- Tensor Data and Precision. Explain what PyTorch tensors hold in a model and why their shape, data type, and device matter. Describe the trade-off between numerical precision and computational efficiency.
- Screening Workflow and Scaffold Split. Trace the high-level PyTorch workflow and explain why this example uses a scaffold-based train/test split with a separate validation set.
- Dataset and DataLoader Roles. Distinguish what the custom MoleculeDataset supplies from what PyTorch DataLoader does, and explain the batching and shuffling choices.
- EGFR Library Screening Workflow. Describe how the trained model would be used to prioritize compounds in the Hinge Binders Library, including checks before ranking.
- Early Recognition and Enrichment. Explain what early recognition means in virtual screening and how an enrichment factor compares a ranked selection with random selection.
- Optimal Enrichment and Base Rate. Explain why optimal EF has a ceiling and why matching that ceiling did not establish that the EGFR model was flawless.
- EGFR Model Quantity Mapping. Explain how the EGFR model’s inputs, output, loss, and gradients correspond to its activity-prediction task.
- Computational Graph Structure. Explain how a computational graph represents a calculation and trace the input, parameter, operation, and loss nodes in the logistic-regression example.
- Backward Pass and Autograd. Describe how the forward pass, backpropagation, and PyTorch Autograd work together to calculate gradients for parameter updates.
- Logistic Unit and Boundary. Describe the weighted inputs and sigmoid activation of a logistic unit, and distinguish its nonlinear output from its linear decision boundary.
- Layered Network Architecture. Describe how artificial neurons are arranged into input, hidden, and output layers, and how information moves through a feedforward network.
- Nonlinear Learned Features. Explain why hidden-layer activations must be nonlinear to model complex relationships, and contrast learned feature extraction with manual feature engineering.
- Kinase Network Structure. Describe how KinaseBinderNN defines its layers and the path from a molecular fingerprint to one prediction.
- Trainable Parameters And Initialization. Explain what the Linear layers learn and why their weights start with suitably scaled random values rather than identical values.
- Activation Functions And Gradients. Distinguish why this architecture uses ReLU rather than saturating activations, while noting ReLU's limitations.
- Dropout During Training. Describe what dropout does during training and why it can reduce overfitting.
- Training batch update cycle. Trace how the network learns from each training batch: move inputs and targets to the device, clear old gradients, make predictions, calculate loss, backpropagate, and update parameters.
- Validation loss and generalization. Explain how the validation pass differs from training and what the training and validation losses reveal about generalization.
What it builds on
- Data Acquisition, Exploration, & Curation (chapter 3) · Assumes familiarity with acquiring and curating bioactivity data.
- Fingerprints: Representing Molecules as Numbers (chapter 2) · Molecular fingerprints as numerical representations.
- Evaluating our Model (chapter 3) · Comparing model predictions with held-out observations.
- Similarity Searching (chapter 2) · Earlier ligand-based screening ranked compounds using similarity searching.
- What is Virtual Screening? (chapter 2) · Virtual screening establishes the task of prioritizing compounds.
- Logistic Regression in Theory (chapter 5) · Introduces logistic regression as a classifier.
- Gradient Descent (chapter 4) · Introduces gradient-based parameter changes.
- Learning from Data (chapter 3) · Provides context for learning from supplied features.
Section by section
Introduction
This introduction previews the chapter’s move from earlier ligand-based machine learning work to applied deep learning with PyTorch and outlines the sections ahead.
8.1 Ligand-based VS with PyTorch
This introduction explains why deep learning is promising for modeling protein–ligand interactions before previewing the section’s kinase and PyTorch example.
- Deep learning modeling promise: Explain that neural networks can learn complex patterns from large amounts of data, making deep learning promising for protein–ligand interaction modeling where traditional computational methods have faced challenges.
Why it matters: This is the motivation for applying deep learning to the protein–ligand problem in this section.
Key terms: deep learning, neural networks, protein-ligand interactions
8.1.1 Protein Kinases
This section explains why EGFR is a useful kinase target and introduces the activity data, inhibitor library, and selectivity considerations for screening it.
- Kinase signaling and opportunity: Explain how protein kinases alter signaling and why they remain important drug targets despite existing approved inhibitors.
Why it matters: It connects the biological effect of kinase inhibition to the opportunity for new drugs. - Other drug target families: Distinguish the receptor families introduced alongside enzyme targets by where they act and the kind of response they produce.
Why it matters: These contrasts keep kinase targets in perspective among other kinds of drug targets. - EGFR cancer target rationale: Explain why EGFR is the chapter’s kinase target and what EGFR-targeting drugs aim to interrupt.
Why it matters: It explains the disease-related purpose of predicting and finding EGFR inhibitors. - EGFR activity data interpretation: Describe what EGFR pIC50 represents, how it relates to potency, and what care the section calls for when assembling activity data.
Why it matters: Correctly interpreting and curating activity values is necessary for a meaningful EGFR prediction task. - Hinge binders and selectivity: Explain why the Enamine hinge binder library is a focused source of possible EGFR inhibitors and why hinge binding alone does not establish EGFR selectivity.
Why it matters: It explains both the library’s appeal for finding hits and the limits of a promising EGFR activity prediction.
Key terms: protein kinases, ATP, phosphate group, kinome, ligand-gated ion channels, GPCRs, tyrosine kinase-linked receptors, nuclear receptors, EGFR, mutations
8.1.2 Our First PyTorch Model
This section introduces PyTorch tensors and walks through preparing data, training, evaluating, and saving a first neural-network model for ligand-based virtual screening.
- Tensor Data and Precision: Explain what PyTorch tensors hold in a model and why their shape, data type, and device matter. Describe the trade-off between numerical precision and computational efficiency.
Why it matters: These properties are often the first things to check when PyTorch code fails or runs inefficiently. - Tensor Operations and Shapes: Describe the tensor operations demonstrated in the section and state the shape rules for matrix multiplication.
Why it matters: Tracking shapes and whether an operation changes the original tensor helps prevent common model-building bugs. - Screening Workflow and Scaffold Split: Trace the high-level PyTorch workflow and explain why this example uses a scaffold-based train/test split with a separate validation set.
Why it matters: The split and workflow determine whether the reported test performance is a meaningful check of generalization. - Dataset and DataLoader Roles: Distinguish what the custom MoleculeDataset supplies from what PyTorch DataLoader does, and explain the batching and shuffling choices.
Why it matters: Separating sample access from batch handling makes the data pipeline easier to manage and training more efficient. - Model Device and Persistence: Describe how the example selects a compute device and saves and reloads its trained model.
Why it matters: Device selection lets the example run without a GPU, while saving the learned state makes the trained model reusable. - Test Performance Interpretation: Describe what the example's test-set errors and prediction plot show, and explain the limit of that evidence for screening decisions.
Why it matters: A promising-looking error score is not the same as evidence that a model is ready to guide real screening.
Key terms: tensor, dtype, shape, device, precision, element-wise, reshape, view, transpose, cat
8.1.3 Enrichment Factors in Virtual Screening
The section shows how to rank an EGFR compound library with a trained model and evaluate whether its ranking finds active compounds early.
- EGFR Library Screening Workflow: Describe how the trained model would be used to prioritize compounds in the Hinge Binders Library, including checks before ranking.
Why it matters: These steps make the ranked shortlist relevant and avoid prioritizing compounds whose activity was already measured. - Early Recognition and Enrichment: Explain what early recognition means in virtual screening and how an enrichment factor compares a ranked selection with random selection.
Why it matters: A screening method is useful when it brings likely hits near the top, not merely when it ranks the full library reasonably well. - Optimal Enrichment and Base Rate: Explain why optimal EF has a ceiling and why matching that ceiling did not establish that the EGFR model was flawless.
Why it matters: An impressive-looking match with optimal EF can be misleading when the test set contains many actives. - Recovered Actives Versus Selected Purity: Distinguish the fraction of all actives recovered at a screening cutoff from the fraction of selected compounds that are active.
Why it matters: Keeping these two fractions distinct prevents a misleading interpretation of screening results. - Enrichment Plots and Global AUC: Describe what an enrichment plot displays and distinguish its early-recognition focus from ROC AUC.
Why it matters: The plot makes early hit recovery visible, while AUC answers a broader ranking question.
Key terms: ligand-based VS, Hinge Binders Library, applicability domain, EGFR pIC50, scoring functions, early recognition, enrichment factor, EF, scoring function, optimal enrichment
8.2 Neural Networks & PyTorch Mechanics
This introduction grounds the upcoming discussion of neural-network mechanics in the EGFR activity-prediction model.
- EGFR Model Quantity Mapping: Explain how the EGFR model’s inputs, output, loss, and gradients correspond to its activity-prediction task.
Why it matters: These concrete quantities give the later neural-network mechanics a drug-discovery context.
Key terms: EGFR, 2,048-bit Morgan fingerprints, pIC50, mean squared error, gradients, weights
8.2.1 Models as Computational Graphs
This section uses computational graphs to explain how PyTorch computes predictions and gradients, then extends a logistic-regression unit into a multilayer neural network.
- Computational Graph Structure: Explain how a computational graph represents a calculation and trace the input, parameter, operation, and loss nodes in the logistic-regression example.
Why it matters: The graph connects a model’s prediction to the calculations used to train it. - Backward Pass and Autograd: Describe how the forward pass, backpropagation, and PyTorch Autograd work together to calculate gradients for parameter updates.
Why it matters: This explains how a model gets the information it needs to improve its parameters. - Gradients and Weight Adjustments: Explain what partial derivatives and the gradient say about loss, and how their direction and the learning rate inform incremental weight changes.
Why it matters: Gradients indicate which way to adjust weights, rather than merely reporting that a prediction was wrong. - Logistic Unit and Boundary: Describe the weighted inputs and sigmoid activation of a logistic unit, and distinguish its nonlinear output from its linear decision boundary.
Why it matters: It prevents confusing a nonlinear probability curve with an ability to separate nonlinearly separable inputs. - Layered Network Architecture: Describe how artificial neurons are arranged into input, hidden, and output layers, and how information moves through a feedforward network.
Why it matters: Architecture describes how a single computational unit becomes a network that produces predictions. - Nonlinear Learned Features: Explain why hidden-layer activations must be nonlinear to model complex relationships, and contrast learned feature extraction with manual feature engineering.
Why it matters: Nonlinearity explains both the network’s capacity for complex patterns and its ability to learn useful features.
Key terms: computational graph, node, edge, loss, forward pass, backward pass, backpropagation, Autograd, backward(), requires_grad
8.2.2 Implementing a Neural Network
This section implements a PyTorch neural network for kinase activity prediction and explains how its layer design, initialization, activations, and dropout affect learning.
- Kinase Network Structure: Describe how KinaseBinderNN defines its layers and the path from a molecular fingerprint to one prediction.
Why it matters: The layer dimensions and forward method determine what data the model can accept and how it produces a prediction. - Trainable Parameters And Initialization: Explain what the Linear layers learn and why their weights start with suitably scaled random values rather than identical values.
Why it matters: Initialization helps neurons learn distinct features without making gradient-based training unstable. - Activation Functions And Gradients: Distinguish why this architecture uses ReLU rather than saturating activations, while noting ReLU's limitations.
Why it matters: An activation's shape affects whether useful gradient updates can reach earlier layers. - Dropout During Training: Describe what dropout does during training and why it can reduce overfitting.
Why it matters: Dropout makes the model less dependent on particular hidden neurons and helps it generalize. - Dropout At Inference: Explain how PyTorch's inverted dropout reconciles training-time zeroing with using every neuron at inference.
Why it matters: It explains why predictions can use all neurons without a separate dropout correction.
Key terms: KinaseBinderNN, nn.Module, __init__, forward, Linear, input_size, weight matrix, bias vector, random initialization, vanishing gradients
8.2.3 Training & Applying our Neural Network
This section explains how to train a PyTorch neural network, monitor validation performance, adjust learning rates, and run inference on test data.
- Training batch update cycle: Trace how the network learns from each training batch: move inputs and targets to the device, clear old gradients, make predictions, calculate loss, backpropagate, and update parameters.
Why it matters: This is the sequence that turns a randomly initialized network into a trained model. - Validation loss and generalization: Explain how the validation pass differs from training and what the training and validation losses reveal about generalization.
Why it matters: A falling training loss alone does not show that the network performs well on held-out data. - Regression loss criterion: Describe what the criterion measures here and why the choice of loss function depends on the task.
Why it matters: The loss function defines the error the training process tries to reduce. - Adam parameter updates: Explain the optimizer’s role and how Adam adapts updates using recent gradients.
Why it matters: Adam determines how calculated gradients become parameter changes during training. - Plateau learning rate schedule: Describe when and how the configured scheduler changes the learning rate.
Why it matters: The schedule responds to stalled validation performance rather than keeping the learning rate fixed. - Test batch inference: Describe how the trained model is applied to test data and what is retained for evaluation.
Why it matters: Saving both predictions and true values makes it possible to evaluate the trained model beyond its training data.
Key terms: forward pass, optimizer.zero_grad(), criterion, loss.backward(), optimizer.step(), backpropagation, model.eval(), torch.no_grad(), validation loss, generalization error
8.3 Summary
Where this is used later
- The Deep Learning Surrogate Model (chapter 9) · The later deep-learning surrogate model also scores compounds for screening.
- Active Learning for Hit Identification: Deep Docking (chapter 9) · Hit identification with limited resources also depends on prioritizing promising compounds.
- How RNNs Process Sequences (chapter 10) · Neural-network computations represented as connected transformations.
- (chapter intro) (chapter 11) · Introduces neural-network models as computational graphs.
- Architectural Components and Information Flow (chapter 11) · Neural network layers, activations, and model implementation.
- Training the Surrogate Model (chapter 9) · Training the deep-learning surrogate model uses the same prediction, loss, gradient, and update cycle.
- Loss Function and Optimization (chapter 11) · The drug-target interaction model also requires a loss function suited to its prediction task.
- Evaluating Pretrained Checkpoints (chapter 11) · Evaluating pretrained checkpoints likewise requires predictions that can be compared with reference values.
Concepts explained in this chapter
- hidden layerA hidden layer is an intermediate neural-network layer whose artificial neurons transform representations between the input and output layers.
- mean squared errorMSE is mean squared error, the average of the squared differences between predicted and observed numerical values.
- DataLoaderPyTorch DataLoader is a utility for batching, optionally shuffling, and iterating over samples from a dataset.
- activation functionAn activation function is a function applied to a neuron's weighted input to produce its output, typically introducing nonlinearity.
- linear layerA linear layer is a neural-network layer that applies an affine transformation to its input using a weight matrix and bias vector.
- neural networkAn artificial neural network is a parameterized computational model composed of interconnected units arranged in layers.
- PyTorchPyTorch is an open-source machine-learning framework for building and training neural networks with tensor computation and automatic differentiation.
- gradientA gradient is a vector containing the partial derivatives of a scalar function with respect to its variables and pointing toward steepest increase.
- backpropagationA backward pass is the reverse computation that propagates derivatives from a loss through a model to obtain parameter gradients.
- forward passA forward pass is the computation that transforms model inputs into predictions by propagating them through the model.
- learning rateA learning rate is an optimization step-size configuration that controls the magnitude of parameter updates during iterative learning.
- tensorA tensor is a multidimensional numerical array that stores data and supports operations for machine-learning computation.
- matrix-matrix multiplicationMatrix multiplication is the arithmetic operation that combines matrices to produce a matrix of weighted linear transformations.
- mean absolute errorMean absolute error is the average of the absolute differences between observed target values and model predictions.
- computation graphA computational graph is a directed graph whose nodes represent computational operations or values and whose edges represent their dependencies.
- ReLUReLU is an activation function that outputs zero for negative inputs and otherwise returns the input value.
- automatic differentiationAutomatic differentiation is the systematic computation of exact derivatives of programs by applying the chain rule to their elementary operations.
- model architectureA neural network architecture is the arrangement and connectivity of layers and neurons that defines a neural network's computational structure.
- neuronAn artificial neuron is a computational unit that transforms weighted inputs, usually with a bias and activation function, into an output.
- sigmoidThe logistic function, or sigmoid, is an S-shaped function that maps real-valued inputs to values between zero and one.
- dropoutDropout is a regularization technique that randomly disables units during training to reduce reliance on particular representations.
- exploding gradientThe exploding gradient problem is a training failure in which gradients grow excessively during backpropagation, causing unstable parameter updates.
- interceptAn intercept is the constant term in a linear model that represents the predicted output when all input features are zero.
- inverted dropoutInverted dropout scales retained activations during training so their expected magnitude matches the full activation signal at inference.
- torch.nn.Modulenn.Module is the PyTorch base class for defining neural-network components with registered parameters, submodules, and computation.
- unstable gradient problemThe unstable gradient problem is the failure of gradients in deep networks to maintain a useful scale as they propagate across layers.
- vanishing gradient problemA vanishing gradient is a training failure mode in which gradients become extremely small during backpropagation, preventing earlier layers from learning effectively.
- weight initializationWeight initialization is the procedure for assigning initial numerical values to a model's learnable weights before training.
- weight matrixA weight matrix is a rectangular array of model parameters whose entries associate input features with class-specific contributions.
- feedforward networkA position-wise feed-forward network is a neural network applied independently to each sequence position, commonly using two linear transformations and a nonlinear activation.
- nonlinear mappingNonlinearity is the property of a function or model in which outputs cannot be expressed as a linear transformation of inputs.
- early recognition problemEarly recognition is the ability of a ranking method to retrieve a large proportion of active compounds within a small initial fraction of a screened collection.
- enrichment factorEnrichment factor is a screening metric that compares the proportion of active compounds found in an early-ranked subset with the proportion expected by random selection.
- parameterA model parameter is a numerical value that governs a model's mapping from inputs to outputs and is typically learned from data.
- regularizationRegularization is a modeling technique that constrains learning to reduce overreliance on particular features and improve performance on unseen data.
- kinaseA protein kinase is an enzyme that transfers phosphate groups to proteins or other biological molecules.
- evaluation datasetA validation set is a held-out collection of examples used during development to assess model performance and guide configuration choices.
- adamAdam is an adaptive stochastic-gradient optimization algorithm that maintains running estimates of gradient moments to update model parameters.
- epochAn epoch is one complete pass through all examples in a machine-learning training dataset.
- generalizationGeneralization is a model's ability to maintain predictive performance on previously unseen data drawn from the relevant distribution.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
