Chapter 12: Transformers for Protein Structure Prediction
This introduction explains why protein structures matter to drug discovery and why the gap between known sequences and experimentally determined structures motivates computational prediction.
After this chapter you will be able to
- Protein Structure in Drug Discovery. Explain why knowing a protein’s three-dimensional structure matters for understanding drug targets and designing treatments.
- Experimental Structure Availability Gap. Describe why experimental methods cannot readily supply structures for all known protein sequences, and distinguish predicted structures from experimental validation.
- Prediction and design directions. Distinguish protein structure prediction, which starts with an amino acid sequence and predicts its three-dimensional structure, from protein design, which starts with a desired structure and seeks a sequence that will fold into it.
- Shared conformational search challenge. Explain why both problems are computationally difficult despite proteins folding quickly in nature.
- Sequence, Shape, and Function. Explain why protein structure prediction starts with an amino acid sequence and why the resulting three-dimensional shape matters for function.
- Four Levels of Structure. Distinguish primary, secondary, tertiary, and quaternary protein structure, and describe which level is the chapter's main prediction focus.
- Helices, Sheets, and Distance. Distinguish alpha-helices from beta-sheets by their geometry and hydrogen bonding, and explain why beta-sheet prediction requires information from distant sequence positions.
- Hydrophobic Core and Backbone Bonds. Explain how the hydrophobic effect and backbone hydrogen bonding jointly shape a protein's tertiary structure.
- Long Range Folding Interactions. Explain why protein structure cannot be predicted from local sequence patterns alone: distant amino acids can interact, while geometry, physics, and evolutionary history constrain the resulting shape.
- Combinatorial Search and Levinthal Paradox. Explain how the numbers of possible backbone conformations and amino-acid sequences make brute-force folding and design impractical, and why rapid real-world folding does not imply random exhaustive search.
- Energy Landscape and Native State. Describe the free-energy landscape as conformations with differing stability, and explain how the thermodynamic hypothesis links a sequence to a low-energy native structure while allowing for the section’s stated caveats.
- Difficulties of Energy Optimization. Explain why finding a minimum-free-energy structure is not a straightforward optimization task, even if the native structure is treated as an energy minimum; describe the limited role of stochastic Monte Carlo moves.
- Sequence to structural signal. Trace how SimpleFold turns an amino acid sequence into per-residue ESM-2 embeddings, and distinguish this input from a hand-built multiple sequence alignment.
- Flow-based coordinate generation. Explain how SimpleFold uses flow matching to generate coordinates from noise and how iterative inference produces a coherent fold.
- Long-range protein interactions. Explain why reasoning only about nearby sequence positions can miss interactions relevant to protein folding. Describe how residues far apart in a sequence can end up close together in the folded structure.
- Transformer and graph attention scope. Distinguish transformer attention from the graph attention networks discussed in Chapter 11. Explain what they share and how the positions each can attend to differ.
- Single Vector Encoding Bottleneck. Explain why an RNN or GRU encoder’s final hidden state is a bottleneck for long protein sequences, including what can happen to information from early residues.
- Decoder Cross-Attention Mechanism. Trace how a decoder uses its current state to select information from all encoder hidden states for one output step, and explain why the selection can change at the next step.
- Encoder-Only Protein Representations. Distinguish a standard encoder–decoder Transformer from the encoder-only architecture used here, and describe what the latter outputs for a protein sequence.
- Self-Attention Across Residues. Explain how self-attention lets each residue use information from other positions, and why that helps with distant interactions compared with sequential RNN processing.
- Scaled Attention Calculation. Trace how residue embeddings become attention weights and updated representations, including why query–key scores are scaled.
- Positional Sequence Information. Justify positional encodings and describe how the sinusoidal version supplies position information.
- Stacked Encoder Block Structure. Describe the two sub-layers of an encoder block, the role of Add & Norm, and what stacking blocks produces.
- Masked sequence learning. Explain how predicting masked amino acids lets a protein language model learn from unlabeled sequences and why the task can reveal biologically meaningful patterns.
- Mixed masking strategy. Describe which positions are selected during training, how their inputs are altered, and why not all selected positions become [MASK] tokens.
- Model prediction path. Trace how token IDs become position-aware contextual representations and then per-position vocabulary predictions in the example model.
- Attention pattern interpretations. Distinguish the local, reference-point, diffuse, and distant-position patterns shown or discussed for attention heads, and explain the point of having multiple heads.
- Antibody Classification Task. Explain how predicting an antibody's virus target differs from masked language modeling, including the labeled input and class output.
- Sequence Pooling And Classification. Trace how the Transformer changes from per-position amino-acid prediction to one class prediction per antibody, including why mean pooling excludes padding.
- Performance And Prediction Interpretation. Interpret the reported accuracy and AUC, explain what near-even class probabilities signal, and qualify conclusions from the held-out test results.
- ESM Scaling and Emergent Structure. Explain how ESM-2 extends masked language modeling to much larger protein datasets and models, and why its sequence-trained representations can reflect protein structure.
- Zero-Shot Mutation Likelihood Ratios. Trace how ESM-2 scores a single amino-acid substitution from masked-position probabilities and interpret the sign of the score.
- Whole-Sequence Plausibility and Validation. Explain how pseudo-perplexity assesses a sequence containing multiple mutations and why model scores still require experimental validation.
What it builds on
- Contact Maps as Structure Representation (chapter 11) · The earlier use of protein structure to represent residue contacts.
- Alternative Architectures: GAT and GIN (chapter 11) · The earlier discussion of graph attention networks.
- Seq2Seq Architecture: Encoding and Decoding Molecules (chapter 10) · The RNN encoder-decoder architecture for molecular sequences.
- How RNNs Process Sequences (chapter 10) · Sequential hidden-state updates in RNNs.
- Binary Classification of CYP3A4 Inhibition (chapter 5) · Assumes basic interpretation of binary classifier outputs and evaluation.
Section by section
Introduction
This introduction explains why protein structures matter to drug discovery and why the gap between known sequences and experimentally determined structures motivates computational prediction.
- Protein Structure in Drug Discovery: Explain why knowing a protein’s three-dimensional structure matters for understanding drug targets and designing treatments.
Why it matters: Protein structure helps connect a target’s sequence and mutations to the behavior relevant to drug design. - Experimental Structure Availability Gap: Describe why experimental methods cannot readily supply structures for all known protein sequences, and distinguish predicted structures from experimental validation.
Why it matters: The shortage of experimentally confirmed structures explains the value—and the limits—of computational prediction. - COVID Vaccine Development Head Start: Explain the introduction’s COVID-19 example of why rapid structural insights matter, while acknowledging what else enabled the vaccine-development head start.
Why it matters: The example shows the practical value of early computational insights without attributing the entire response to modeling.
Key terms: three-dimensional protein structures, drug-target binding affinity, structure determines function, experimental structure determination, Protein Data Bank (PDB), protein sequences, computationally predicted structures, experimental validation, SARS-CoV-2, spike protein
12.1 Conjoined Problems: Structure Prediction & Protein Design
This section frames protein structure prediction and protein design as inverse problems that share a vast conformational search challenge.
- Prediction and design directions: Distinguish protein structure prediction, which starts with an amino acid sequence and predicts its three-dimensional structure, from protein design, which starts with a desired structure and seeks a sequence that will fold into it.
Why it matters: The two directions clarify whether a model is predicting a fold or designing a protein to achieve one. - Shared conformational search challenge: Explain why both problems are computationally difficult despite proteins folding quickly in nature.
Why it matters: This contrast explains why folding and design call for approaches beyond trying every possible shape.
Key terms: protein structure prediction, protein design, amino acid sequence, three-dimensional structure, conformations, exhaustive search, functional structures
12.1.1 Protein Structure Essentials
This section explains how amino acid sequences form functional three-dimensional protein structures and what a structure-prediction model must capture.
- Sequence, Shape, and Function: Explain why protein structure prediction starts with an amino acid sequence and why the resulting three-dimensional shape matters for function.
Why it matters: It connects the sequence given to a model with the biological reason to predict its shape. - Four Levels of Structure: Distinguish primary, secondary, tertiary, and quaternary protein structure, and describe which level is the chapter's main prediction focus.
Why it matters: The levels clarify whether someone means a sequence, a local pattern, one folded chain, or a multi-chain complex. - Backbone and Sidechain Chemistry: Describe the repeating protein backbone, what varies among amino acids, and how sidechain properties relate to placement in a folded protein.
Why it matters: The sequence carries chemical differences that help explain where residues end up in a fold. - Helices, Sheets, and Distance: Distinguish alpha-helices from beta-sheets by their geometry and hydrogen bonding, and explain why beta-sheet prediction requires information from distant sequence positions.
Why it matters: The contrast shows why recognizing local patterns alone cannot account for all protein structure. - Hydrophobic Core and Backbone Bonds: Explain how the hydrophobic effect and backbone hydrogen bonding jointly shape a protein's tertiary structure.
Why it matters: It explains why a compact fold must accommodate both sidechain chemistry and the backbone. - Structure Prediction Modeling Demands: Describe the different scales of information a structure-prediction model must capture and why exhaustive conformational search is not practical.
Why it matters: It summarizes why sequence-to-structure prediction demands more than local pattern recognition or brute-force search.
Key terms: AA sequence, three-dimensional structure, active site, primary structure, secondary structure, tertiary structure, quaternary structure, peptide bonds, backbone, sidechain
12.1.2 Why Protein Folding is a Hard Modeling Problem
This section explains why predicting and designing protein structures are difficult, from vast search spaces and rugged energy landscapes to the limits of earlier methods and the appeal of learned sequence-to-structure patterns.
- Long Range Folding Interactions: Explain why protein structure cannot be predicted from local sequence patterns alone: distant amino acids can interact, while geometry, physics, and evolutionary history constrain the resulting shape.
Why it matters: This explains why a useful folding model must account for relationships across the sequence. - Forward and Inverse Problems: Distinguish structure prediction, which starts with an amino-acid sequence and asks for its three-dimensional structure, from protein design, which starts with a desired structure and seeks a sequence that will adopt it.
Why it matters: Keeping the two directions distinct clarifies which space—structures or sequences—each problem searches. - Combinatorial Search and Levinthal Paradox: Explain how the numbers of possible backbone conformations and amino-acid sequences make brute-force folding and design impractical, and why rapid real-world folding does not imply random exhaustive search.
Why it matters: The search-space argument shows why simply trying every fold or every sequence cannot solve these problems. - Energy Landscape and Native State: Describe the free-energy landscape as conformations with differing stability, and explain how the thermodynamic hypothesis links a sequence to a low-energy native structure while allowing for the section’s stated caveats.
Why it matters: The landscape picture explains both why low-energy structures are attractive prediction targets and why that picture needs qualifications. - Difficulties of Energy Optimization: Explain why finding a minimum-free-energy structure is not a straightforward optimization task, even if the native structure is treated as an energy minimum; describe the limited role of stochastic Monte Carlo moves.
Why it matters: It shows why stating an energy-minimization objective does not by itself provide a practical folding solution. - Historical and Learned Prediction Approaches: Contrast molecular dynamics, homology modeling, and modern deep learning at the level described here: what each draws on and what limitation or advantage motivates the shift.
Why it matters: This contrast explains what deep learning changes without claiming that earlier approaches solved every folding case.
Key terms: amino acids, three-dimensional shape, local patterns, forward problem, protein structure prediction, inverse problem, protein design, phi and psi torsion angles, backbone conformations, Levinthal's paradox
12.2 SimpleFold: An End-to-End Example
This section traces how SimpleFold turns an amino acid sequence into a predicted protein structure and explains how to interpret the result.
- Sequence to structural signal: Trace how SimpleFold turns an amino acid sequence into per-residue ESM-2 embeddings, and distinguish this input from a hand-built multiple sequence alignment.
Why it matters: This identifies where SimpleFold gets useful protein information before generating coordinates. - Flow-based coordinate generation: Explain how SimpleFold uses flow matching to generate coordinates from noise and how iterative inference produces a coherent fold.
Why it matters: It explains how SimpleFold turns sequence-derived information into a three-dimensional prediction. - Prediction confidence and limits: Describe what SimpleFold outputs alongside coordinates and why its predictions need caution on some targets.
Why it matters: A predicted fold is more useful when its uncertain regions and likely failure cases are recognized.
Key terms: amino acid sequence, ESM-2, per-residue embedding, multiple sequence alignment (MSA), flow matching, velocity field, iterative refinement, all-atom coordinates, per-residue confidence estimate, evolutionary signal
12.3 Protein Language Models
This section explains why protein sequences need long-range attention and how transformer attention extends the graph attention used in Chapter 11.
- Long-range protein interactions: Explain why reasoning only about nearby sequence positions can miss interactions relevant to protein folding. Describe how residues far apart in a sequence can end up close together in the folded structure.
Why it matters: It explains why a model of protein sequences needs a way to connect distant residues. - Transformer and graph attention scope: Distinguish transformer attention from the graph attention networks discussed in Chapter 11. Explain what they share and how the positions each can attend to differ.
Why it matters: The distinction shows what transformers add to the attention mechanism already used for molecular and protein graphs.
Key terms: AAs, beta sheets, disulfide bonds, global optimization problem, graph attention networks (GATs), transformers, attention weights, weighted aggregation, global attention
12.3.1 Why Transformers? From Local to Global Attention
The section explains why compressing a long protein sequence into one recurrent hidden state loses information and how cross-attention lets a decoder draw on all encoder states.
- Single Vector Encoding Bottleneck: Explain why an RNN or GRU encoder’s final hidden state is a bottleneck for long protein sequences, including what can happen to information from early residues.
Why it matters: This limitation motivates giving the decoder access to more than the encoder’s final state. - Intermediate Hidden State Information: Describe what intermediate encoder hidden states retain and why keeping them is useful instead of relying only on the final state.
Why it matters: The retained states give attention access to context from different parts of a protein. - Decoder Cross-Attention Mechanism: Trace how a decoder uses its current state to select information from all encoder hidden states for one output step, and explain why the selection can change at the next step.
Why it matters: Cross-attention replaces a single, unchanging input summary with information selected for each output step.
Key terms: encoding bottleneck, hidden state, context vector, GRUs, local context, encoder hidden states, cross-attention, query (Q), keys (K), values (V)
12.3.2 The Encoder-Only Transformer Architecture
Explains how an encoder-only Transformer turns a protein sequence into contextual residue representations using self-attention, positional information, and stacked encoder blocks.
- Encoder-Only Protein Representations: Distinguish a standard encoder–decoder Transformer from the encoder-only architecture used here, and describe what the latter outputs for a protein sequence.
Why it matters: This distinguishes representing an existing protein from generating a new sequence. - Self-Attention Across Residues: Explain how self-attention lets each residue use information from other positions, and why that helps with distant interactions compared with sequential RNN processing.
Why it matters: Residues far apart in a protein sequence may need to influence one another’s representations. - Scaled Attention Calculation: Trace how residue embeddings become attention weights and updated representations, including why query–key scores are scaled.
Why it matters: The calculation shows how a model turns learned relevance between residues into contextual features. - Multiple Attention Heads: Explain what separate attention heads contribute and how their outputs become one representation.
Why it matters: Different heads give the encoder more than one way to find useful residue relationships. - Positional Sequence Information: Justify positional encodings and describe how the sinusoidal version supplies position information.
Why it matters: Protein sequence order matters, so attention needs a way to distinguish where residues occur. - Stacked Encoder Block Structure: Describe the two sub-layers of an encoder block, the role of Add & Norm, and what stacking blocks produces.
Why it matters: The block structure connects individual attention calculations to a deep protein representation model.
Key terms: encoder-only transformer, encoder, decoder, embeddings, self-attention, queries, keys, values, long-range dependencies, scaled dot-product attention
12.3.3 Training a Small Protein Language Model
This section explains how a small protein language model learns from masked amino acids and how to interpret its predictions and attention patterns.
- Masked sequence learning: Explain how predicting masked amino acids lets a protein language model learn from unlabeled sequences and why the task can reveal biologically meaningful patterns.
Why it matters: It explains what the model learns before it is used for a task such as antibody classification. - Mixed masking strategy: Describe which positions are selected during training, how their inputs are altered, and why not all selected positions become [MASK] tokens.
Why it matters: The mixture helps the model produce useful representations when given ordinary, unmasked sequences. - Model prediction path: Trace how token IDs become position-aware contextual representations and then per-position vocabulary predictions in the example model.
Why it matters: It connects the encoder’s learned representations to the amino-acid predictions used for training. - Interpreting residue predictions: Describe what the trained model’s masked-residue examples show, including why an incorrect prediction can still be informative.
Why it matters: The examples show what learned sequence patterns can look like in the model’s predictions. - Attention pattern interpretations: Distinguish the local, reference-point, diffuse, and distant-position patterns shown or discussed for attention heads, and explain the point of having multiple heads.
Why it matters: Reading the patterns helps explain why multiple attention heads can capture more than one kind of sequence context. - Pretraining evaluation limits: State what the masked-language-modeling task accomplishes on its own and identify the evaluation and data cautions named in the problem frame.
Why it matters: It separates success at pretraining from success on a downstream task and flags misleading evaluation.
Key terms: masked language modeling, self-supervised learning, amino acids, [MASK] token, masking strategy, unmasked sequences, embedding layer, positional encoding, transformer encoder, language modeling head
12.3.4 Downstream Applications: Antibody Classification
This section adapts a protein Transformer to classify antibody sequences by virus target, then interprets its predictions, performance, and limitations.
- Antibody Classification Task: Explain how predicting an antibody's virus target differs from masked language modeling, including the labeled input and class output.
Why it matters: The labels change both what the model learns to predict and how its predictions can aid screening. - Sequence Pooling And Classification: Trace how the Transformer changes from per-position amino-acid prediction to one class prediction per antibody, including why mean pooling excludes padding.
Why it matters: Pooling makes variable-length antibody sequences usable for a single sequence-level decision. - Classifier Training Choices: Describe how classifier training differs from masked language model training and why the example uses a relatively small model.
Why it matters: The training setup must match sequence-level labels while accounting for the small amount of labeled data. - Performance And Prediction Interpretation: Interpret the reported accuracy and AUC, explain what near-even class probabilities signal, and qualify conclusions from the held-out test results.
Why it matters: Screening decisions depend on understanding both the model's ranking ability and the uncertainty in its reported performance. - Error Sources And Checks: Describe how uncertainty, the confusion matrix, and sequence-length comparisons can reveal classifier limitations, including possible dataset confounders.
Why it matters: Investigating errors helps distinguish useful antibody patterns from dataset shortcuts or systematic failures. - Pretraining Transfer Rationale: Explain why starting with a pre-trained protein language model could improve this limited-data classification task.
Why it matters: Pretraining offers a way to use broad protein-sequence knowledge when target-labeled antibodies are scarce.
Key terms: antibody heavy chain, supervised classification, masked language modeling, virus target, language modeling head, mean pooling, attention mask, classification head, padding tokens, cross-entropy loss
12.3.5 The Evolutionary Scale Modeling (ESM) Family of Models
This section explains how scaled protein language models provide reusable protein representations for classification, mutation scoring, and structure prediction.
- ESM Scaling and Emergent Structure: Explain how ESM-2 extends masked language modeling to much larger protein datasets and models, and why its sequence-trained representations can reflect protein structure.
Why it matters: It explains why a pretrained sequence model can be useful for protein tasks beyond predicting missing amino acids. - Pretrained Embeddings for Classification: Describe how ESM-2 per-residue embeddings can support antibody classification and how to evaluate that transfer without leakage.
Why it matters: Pretrained embeddings make small labeled datasets more useful, but near-duplicate antibodies can make an evaluation look better than it is. - Zero-Shot Mutation Likelihood Ratios: Trace how ESM-2 scores a single amino-acid substitution from masked-position probabilities and interpret the sign of the score.
Why it matters: This score can help rank variants for testing before committing to laboratory assays. - Mutation Landscape for Prioritization: Describe how an all-substitutions mutation landscape reveals constrained and tolerant positions and guides the choice of candidates to test.
Why it matters: A landscape helps narrow a large mutation search to positions and substitutions worth examining further. - Whole-Sequence Plausibility and Validation: Explain how pseudo-perplexity assesses a sequence containing multiple mutations and why model scores still require experimental validation.
Why it matters: Checking a complete variant helps prioritize combinations while keeping model plausibility separate from measured performance. - Language Models in Folding Systems: Distinguish how ESMFold and SimpleFold use protein language model representations in their structure-prediction pipelines.
Why it matters: It shows how a pretrained sequence representation becomes a component of a practical structure-prediction system.
Key terms: ESM-2, encoder-only Transformer, masked language modeling, emergent structure, rotary position embeddings, per-residue embeddings, mean pooling, classification head, feature extraction, fine-tuning
12.4 Summary
Where this is used later
- Case Study 3: Protein Therapeutic Design - A Multi-Model Workflow (chapter 13) · The protein therapeutic design case study draws on the structure-to-sequence direction.
- Platform Overview (chapter 13) · Frames protein structure prediction and protein design as related problems.
- Integrating Multimodal Representations (chapter 13) · Prior discussion of the ESM family of protein models.
- Case Study 2: Virtual Screening at Billion-Compound Scale (chapter 13) · Protein language models and the ESM family.
Concepts explained in this chapter
- attentionAn attention mechanism computes input-dependent weights that determine how strongly elements of a sequence or representation influence one another.
- transformer architectureA transformer architecture is a neural-network architecture that models relationships among sequence elements using attention and position-wise transformations, potentially with encoder and decoder stacks.
- masked language modelingMasked language modeling is a self-supervised technique that trains a model to predict tokens hidden or replaced within an input sequence.
- self-supervised learningSelf-supervised learning trains a model using supervisory signals or prediction targets derived from the input data itself rather than external labels.
- cross-attentionCross-attention is an attention mechanism in which queries from one sequence or representation attend to keys and values derived from another.
- encoder-only transformerAn encoder-only transformer is a transformer consisting of encoder layers that produces contextual representations without an autoregressive decoder.
- multi-head attentionMulti-head attention is an attention mechanism that combines multiple independently parameterized attention operations to capture different relationships in a sequence.
- positional embeddingA positional encoding is a vector representation of a token's location that is added to its content embedding to provide sequence-order information.
- ESM-2ESM-2 is a protein language model trained on large collections of amino acid sequences to learn sequence representations and predict mutation effects.
- SimpleFoldSimpleFold is a protein structure prediction model that generates structured protein conformations from noisy representations using flow matching.
- amino acid sequenceProtein primary structure is the linear order of amino acid residues in a protein chain.
- protein folding problemProtein folding is the process by which an amino acid sequence adopts a three-dimensional structure.
- log-ratioA log-likelihood ratio is the logarithm of the ratio between two likelihoods, used here to compare the plausibility of mutated and reference protein sequences.
- pseudo-perplexityPseudo-perplexity is an evaluation metric for masked language models that aggregates the model's conditional uncertainty over tokens masked in a sequence.
- zero-shot learningZero-shot transfer is the application of a model to a task or modality without task-specific labeled examples for that application.
- protein structure predictionProtein structure prediction is the computational inference of a protein's three-dimensional structure from its amino acid sequence or related data.
- ESMEvolutionary Scale Modeling is a family of protein language models trained on large collections of amino acid sequences.
- ESMFoldESMFold is a protein structure prediction model that uses protein-language-model embeddings to predict three-dimensional structure without generating a multiple sequence alignment.
- Rotary Position EmbeddingA rotary position embedding encodes relative positional information by rotating query and key representations within attention computations.
- contact mapA contact map is a representation indicating which pairs of residues or atoms in a biomolecule are spatially close or interacting.
- epistasisEpistasis is the dependence of a mutation's effect on the presence of other mutations in the same sequence.
- low-rank adaptationLoRA is a parameter-efficient fine-tuning method that learns low-rank updates while keeping most pretrained weights fixed.
- embeddingAn embedding is a learned numerical vector representation of an object that captures properties relevant to a task.
- encoderAn encoder is a model component that transforms an input into a lower-dimensional representation while retaining selected information.
- feedforward networkA position-wise feed-forward network is a neural network applied independently to each sequence position, commonly using two linear transformations and a nonlinear activation.
- layer normalizationLayer normalization normalizes features within each individual sample and applies learned scale and shift parameters.
- residual connectionA residual connection adds a layer's input representation to that layer's transformed output.
- scaled-dot product attentionScaled dot-product attention computes attention weights from query-key dot products divided by the square root of the key dimension and applies them to value vectors.
- sinusoidal positional encodingA sinusoidal position encoding represents sequence positions with fixed sine and cosine functions added to model inputs.
- protein language modelA protein language model is a model trained on protein sequences to learn sequence representations or predict biologically meaningful sequence patterns.
- protein structureProtein structure is the three-dimensional arrangement of a protein's atoms and constituent chains.
- hydrophobic effectThe hydrophobic effect is the tendency of nonpolar molecular groups to associate away from water, often causing them to become buried within biological structures.
- protein designProtein design is the creation or selection of amino acid sequences intended to adopt a specified three-dimensional structure and satisfy desired stability or functional constraints.
- cross entropy lossCross-entropy is a loss function that measures the discrepancy between a target probability distribution and a predicted probability distribution.
- [CLS] tokenA CLS token is a special Transformer input token whose contextual representation can summarize an entire sequence for classification.
- antibodyAn antibody is an immune-system protein that specifically recognizes and can help neutralize a pathogen or other antigen.
- antibody heavy chainA heavy chain is one of the larger polypeptide chains forming an antibody and contributing to its antigen-binding and effector properties.
- antigenAn antigen is a molecule or molecular region specifically recognized by an antibody or other immune receptor.
- area under the ROC curveAUC is the area under the receiver operating characteristic curve and measures ranking-based discrimination between classes.
- attention_maskAn attention mask is a tensor or rule that marks which sequence positions should contribute to attention or aggregation computations.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
