Multimodal AI Predicts Clinical Outcomes of Drug Combinations from Preclinical Data

Combination therapies are widely used to treat diseases, from hypertension to cancer and infectious diseases. Computational models typically use structural data or target profiles to identify drug combinations with high efficacy and low toxicity, yet often overlook complex biological interactions. Addressing these gaps requires integrating diverse modalities—such as structural data, pathways, cell viability, and transcriptomics—to capture the complexity of clinical drug combinations from multimodal preclinical data.

Here, we introduce Madrigal, a unified multimodal AI model for predicting drug combinations across 953 clinical outcomes and 21,842 compounds, including approved drugs and novel compounds in development. Madrigal uses a bottleneck module with modality alignment to transfer information across modality-specific encoders and address the all-stage missing modality challenge, a problem that plagues multimodal learning in medicine.

Madrigal outperforms single-modality methods and state-of-the-art models in predicting adverse drug interactions. We demonstrate Madrigal for identifying transporter-mediated drug interactions, drug combination virtual screening of anti-cancer combinations, and identifying polypharmacy needs for type II diabetes and metabolic dysfunction-associated steatohepatitis (MASH). Madrigal ranks resmetirom, the first and only FDA-approved drug for metabolic dysfunction-associated steatohepatitis (MASH), among the safest candidates. Madrigal applies to individualized cancer therapies, as demonstrated in primary acute myeloid leukemia samples and patient-derived xenograft models, where it predicts personalized drug combination efficacy based on genomic profiles. By integrating Madrigal with a large language model, users can describe clinical outcomes in natural language. Madrigal offers a multimodal approach to designing safe combination therapies.

Publication

Multimodal AI predicts clinical outcomes of drug combinations from preclinical data
Yepeng Huang, Xiaorui Su, Varun Ullanat, Ivy Liang, Lindsay Clegg, Damilola Olabode, Nicholas Ho, Bino John, Megan Gibbs, Marinka Zitnik
In Review 2024 [bioRxiv]

@article{huang2024madrigal,
  title={Multimodal AI predicts clinical outcomes of drug combinations from preclinical data},
  author={Huang, Yepeng and Su, Xiaorui and Ullanat, Varun and Liang, Ivy and Clegg, Lindsay and Olabode, Damilola and  Ho, Nicholas and John, Bino and Gibbs, Megan and Zitnik, Marinka},
  journal={biorxiv},
  url={},
  year={2024}
}

Code and Data Availability

Pytorch implementation of PocketGen is available in the GitHub repository. Datasets are also available at Harvard Dataverse repository.

Authors

Latest News

Dec 2024:   Unified Clinical Vocabulary Embeddings

New paper: A unified resource provides a new representation of clinical knowledge by unifying medical vocabularies. (1) Phenotype risk score analysis across 4.57 million patients, (2) Inter-institutional clinician panels evaluate alignment with clinical knowledge across 90 diseases and 3,000 clinical codes.

Dec 2024:   SPECTRA in Nature Machine Intelligence

Are biomedical AI models truly as smart as they seem? SPECTRA is a framework that evaluates models by considering the full spectrum of cross-split overlap: train-test similarity. SPECTRA reveals gaps in benchmarks for molecular sequence data across 19 models, including LLMs, GNNs, diffusion models, and conv nets.

Nov 2024:   Ayush Noori Selected as a Rhodes Scholar

Congratulations to Ayush Noori on being named a Rhodes Scholar! Such an incredible achievement!

Nov 2024:   PocketGen in Nature Machine Intelligence

Oct 2024:   Activity Cliffs in Molecular Properties

Oct 2024:   Knowledge Graph Agent for Medical Reasoning

Sep 2024:   Three Papers Accepted to NeurIPS

Exciting projects include a unified multi-task time series model, a flow-matching approach for generating protein pockets using geometric priors, and a tokenization method that produces invariant molecular representations for integration into large language models.

Sep 2024:   TxGNN Published in Nature Medicine

Aug 2024:   Graph AI in Medicine

Excited to share a new perspective on Graph Artificial Intelligence in Medicine in Annual Reviews.

Aug 2024:   How Proteins Behave in Context

Harvard Medicine News on our new AI tool that captures how proteins behave in context. Kempner Institute on how context matters for foundation models in biology.

Jul 2024:   PINNACLE in Nature Methods

PINNACLE contextual AI model is published in Nature Methods. Paper. Research Briefing. Project website.

Jul 2024:   Digital Twins as Global Health and Disease Models of Individuals

Paper on digitial twins outlining strategies to leverage molecular and computational techniques to construct dynamic digital twins on the scale of populations to individuals.

Jul 2024:   Three Papers: TrialBench, 3D Structure Design, LLM Editing

Jun 2024:   TDC-2: Multimodal Foundation for Therapeutics

The Commons 2.0 (TDC-2) is an overhaul of Therapeutic Data Commons to catalyze research in multimodal models for drug discovery by unifying single-cell biology of diseases, biochemistry of molecules, and effects of drugs through multimodal datasets, AI-powered API endpoints, new tasks and benchmarks. Our paper.

May 2024:   Broad MIA: Protein Language Models

Apr 2024:   Biomedical AI Agents

Mar 2024:   Efficient ML Seminar Series

We started a Harvard University Efficient ML Seminar Series. Congrats to Jonathan for spearheading this initiative. Harvard Magazine covered the first meeting focusing on LLMs.

Zitnik Lab  ·  Artificial Intelligence in Medicine and Science  ·  Harvard  ·  Department of Biomedical Informatics