Journal article · 2026 · Open access
A clinical multimodal vision language foundation model with fine-grain explainability
Nature Medicine, 32(11), 4208–4220 · Published 29 October 2026 · DOI 10.1038/s41591-026-04655-5
In brief
“A clinical multimodal vision language foundation model with fine-grain explainability” is a 2026 journal article by Yang Bai and 48 co-authors, published in Nature Medicine. OpenAlex counts 2 citations as of 16 November 2026, which puts it in the top 5% of works published in 2026 by citations. It is open access under a CC BY-NC-ND licence: the full text is free to read at Nature Medicine.
- Citations
- 2
- Most cited in
- 2026 (2)
- Field-weighted impact
- 2.94
- References
- 82
- Authors
- 49
Abstract
Despite their considerable promise, many current medical foundation models remain modality specific and lack transparent reasoning, limiting clinical adoption. To address this gap, we introduce Explainable Vision Language Foundation Model for Medicine (EVLF-FM), a multimodal vision language foundation model that combines broad diagnostic capabilities with fine-grained explainability. The development and testing of EVLF-FM encompassed more than 1.3 million total samples from 18 global datasets across 10 imaging modalities related to six clinical specialties: dermatology, hepatology, ophthalmology, pathology, pulmonology and radiology. External validation employed 13,443 independent test samples from 17 additional datasets across 10 imaging modalities. EVLF-FM supports disease diagnosis, visual question answering and pixel-level visual grounding with reasoning capabilities. In internal validation for disease diagnostics, EVLF-FM achieved the highest average accuracy, 0.895, outperforming leading generalist and specialist models. In medical visual grounding, EVLF-FM also achieved reasonable performance across nine modalities with average accuracy at 0.5 intersection over union (Acc@0.5) of 0.842. External testing further confirmed consistent zero-shot and few-shot performance, with competitive F1 scores despite a smaller model size. A hybrid training strategy combining supervised and visual reinforcement fine-tuning enables EVLF-FM to generate clinically interpretable explanations aligned with visual evidence. Human evaluation of these reasoning outputs using the S.C.O.R.E. framework yielded a high overall mean score of 4.25/5, supporting their clinical interpretability. Current limitations include the need for additional validation across diverse healthcare settings. Prospective clinical studies are required to evaluate the safety, effectiveness and clinical utility of EVLF-FM before routine clinical deployment.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Bai, Y., Cheng, H., Zhou, Y., Zhou, J., Thirunavukarasu, A. J., Ke, Y., Yao, J., Fukutsu, K., Quek, C. W. N., Hong, A. S. Y., Gutiérrez, L., Berni, A., Teo, Z. L., Ting, D. S. J., Soetikno, B. T., Nielsen, C. S., Elze, T., Li, Z., Sharma, S., . . . Ting, D. S. W. (2026). A clinical multimodal vision language foundation model with fine-grain explainability. Nature Medicine, 32(11), 4208–4220. https://doi.org/10.1038/s41591-026-04655-5
MLA 9
Bai, Yang, et al. “A clinical multimodal vision language foundation model with fine-grain explainability.” Nature Medicine, vol. 32, no. 11, 2026, pp. 4208–4220. https://doi.org/10.1038/s41591-026-04655-5.
Chicago
Bai, Yang, Haoran Cheng, Yang Zhou, Jun Zhou, Arun James Thirunavukarasu, Yuhe Ke, Jie Yao, et al. 2026. “A clinical multimodal vision language foundation model with fine-grain explainability.” Nature Medicine 32 (11): 4208–4220. https://doi.org/10.1038/s41591-026-04655-5.
Harvard
Bai, Y. et al. (2026) ‘A clinical multimodal vision language foundation model with fine-grain explainability’, Nature Medicine, 32(11), pp. 4208–4220. doi:10.1038/s41591-026-04655-5.
IEEE
Y. Bai et al., “A clinical multimodal vision language foundation model with fine-grain explainability,” Nature Medicine, vol. 32, no. 11, pp. 4208–4220, 2026, doi: 10.1038/s41591-026-04655-5.
BibTeX
@article{bai2026clinical,
title = {{A clinical multimodal vision language foundation model with fine-grain explainability}},
author = {Bai, Yang and Cheng, Haoran and Zhou, Yang and Zhou, Jun and Thirunavukarasu, Arun James and Ke, Yuhe and Yao, Jie and Fukutsu, Kanae and Quek, Chrystie Wan Ning and Hong, Ashley Shuen Ying and Gutiérrez, Laura and Berni, Alessandro and Teo, Zhen Ling and Ting, Darren Shu Jeng and Soetikno, Brian T. and Nielsen, Christopher S and Elze, Tobias and Li, Zengxiang and Sharma, Sumit and Ruthmann, Nicholas P. and Linh, Le Dinh and Chan, Hiok Hong and Koh, Victor and Tan, Marcus Chun Jin and Li, Kelvin Z. and Yip, Leonard and Cheng, Ching Yu and Tham, Yih Chung and Tan, Gavin Siew Wei and Schmetterer, Leopold and others},
journal = {Nature Medicine},
year = {2026},
volume = {32},
number = {11},
pages = {4208--4220},
doi = {10.1038/s41591-026-04655-5},
url = {https://doi.org/10.1038/s41591-026-04655-5}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- vision-language foundation models
- diagnostic imaging
- explainable AI
- visual grounding
- disease diagnosis
- visual question answering
- visual reinforcement learning
- clinical interpretability
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Bai, Y., Cheng, H., Zhou, Y., Zhou, J., Thirunavukarasu, A. J., Ke, Y., Yao, J., Fukutsu, K., Quek, C. W. N., Hong, A. S. Y., Gutiérrez, L., Berni, A., Teo, Z. L., Ting, D. S. J., Soetikno, B. T., Nielsen, C. S., Elze, T., Li, Z., Sharma, S., . . . Ting, D. S. W. (2026). A clinical multimodal vision language foundation model with fine-grain explainability. Nature Medicine, 32(11), 4208–4220. https://doi.org/10.1038/s41591-026-04655-5
How many times has this paper been cited?
2 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Nature Medicine: sign in and open the PDF from this page.
Who wrote this paper?
Yang Bai, Haoran Cheng, Yang Zhou, Jun Zhou, Arun James Thirunavukarasu, Yuhe Ke, Jie Yao, Kanae Fukutsu, Chrystie Wan Ning Quek, Ashley Shuen Ying Hong, Laura Gutiérrez, Alessandro Berni and 37 more.
What is the DOI of this paper?
10.1038/s41591-026-04655-5, permanently at https://doi.org/10.1038/s41591-026-04655-5. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.