Multimodal data combines observations from distinct data modalities so that evidence from one representation can inform analysis of another. In drug discovery, these modalities have different structures, scales, and meanings, making cross-modal representation a central engineering challenge. section 13.2.4
Common drug-discovery modalities include:
- Protein sequences represented as amino-acid strings. section 13.2.4
- Molecular structures represented as graphs of atoms and bonds. section 13.2.4
- Cellular images represented as three-dimensional pixel arrays. section 13.2.4
- Gene-expression profiles represented as high-dimensional numerical vectors. section 13.2.4
Building multimodal datasets requires harmonization: entity linking resolves identifiers such as JAK2 and Entrez Gene ID 3717, ontology mapping aligns disease vocabularies, and quality scores represent evidence reliability. section 13.2.4


