liveBook · Manning

Multimodal data

As explained in Build AI Drug Discovery Pipelines

A multimodal dataset is a collection combining observations from two or more distinct data modalities.

Multimodal data combines observations from distinct data modalities so that evidence from one representation can inform analysis of another. In drug discovery, these modalities have different structures, scales, and meanings, making cross-modal representation a central engineering challenge. section 13.2.4

Common drug-discovery modalities include:

  • Protein sequences represented as amino-acid strings. section 13.2.4
  • Molecular structures represented as graphs of atoms and bonds. section 13.2.4
  • Cellular images represented as three-dimensional pixel arrays. section 13.2.4
  • Gene-expression profiles represented as high-dimensional numerical vectors. section 13.2.4

Building multimodal datasets requires harmonization: entity linking resolves identifiers such as JAK2 and Entrez Gene ID 3717, ontology mapping aligns disease vocabularies, and quality scores represent evidence reliability. section 13.2.4

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains modality representations and dataset harmonization through entity linking, ontology mapping, and relationship confidence scoring.
Examples
Uses protein sequences, molecular graphs, cellular images, gene-expression vectors, disease ontologies, and JAK2 identifier matching.
What you can do after
Evaluate how heterogeneous biological observations can be represented, aligned, and assigned evidence confidence across data sources.
How it connects
Connects multimodal data with data harmonization, entity linking, ontology mapping, knowledge graphs, and quality scoring.

Also explained in

Compare how 3 books explain multimodal data →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.