liveBook · Manning

Multimodal foundation model

As explained in Build AI Drug Discovery Pipelines

A multimodal biological foundation model jointly represents biological information from multiple modalities such as sequences, structures, expression measurements, and phenotypes.

A multimodal foundation model learns shared representations across biological modalities such as DNA sequence, RNA expression, protein structure, cellular images, literature, and phenotypes. Instead of treating each prediction problem separately, it links related biological entities in a common representation space, supporting transfer between modalities and across biological scales. section 13.2.4 section 13.4.4

Core design ideas include:

  • Use self-supervised objectives, such as masked-region prediction or matching images from the same perturbation, to learn biologically meaningful features without exhaustive labels. section 13.2.4
  • Map equivalent biological entities near one another even when they appear as sequences, expression patterns, or structures, enabling knowledge transfer between modalities. section 13.2.4 section 13.4.4
  • Extend modeling across scales, from molecules and proteins through cells, tissues, and organism-level physiology. section 13.2.4 section 13.4.4

LucaOne

LucaOne jointly pretrains on gene DNA sequences, RNA expression patterns, and protein structures. A function learned from protein structure analysis can therefore inform predictions about RNA expression, illustrating zero-shot transfer through a shared representation space. section 13.2.4 section 13.4.4

Broader integration

ProCyon integrates sequences, structures, scientific-literature text, knowledge graphs, and functional annotations for tasks including protein function, drug-target interaction, and clinical outcome prediction. Evo 2 instead emphasizes genome-scale sequence modeling across millions of tokens. section 13.4.4 section 13.4.4

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains shared spaces, self-supervision, multi-task pretraining, and efficient attention, but stops short of a complete training recipe.
Examples
Uses Phenom, LucaOne, ProCyon, and Evo 2 to illustrate imaging, cross-modal, multi-domain, and genome-scale modeling.
What you can do after
Evaluate/understand how shared biological representations enable zero-shot transfer and broader prediction across modalities and scales.
How it connects
Connects DNA sequences, RNA expression, protein structures, cellular imaging, phenotypes, knowledge graphs, and drug-target interaction prediction.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.