Journal article · 2026 · Open access
General-purpose large language models outperform specialized clinical AI tools on medical benchmarks
Nature Medicine, 32(7), 2405–2409 · Published 12 June 2026 · DOI 10.1038/s41591-026-04431-5
In brief
“General-purpose large language models outperform specialized clinical AI tools on medical benchmarks” is a 2026 journal article by Krithik Vishwanath and 15 co-authors, published in Nature Medicine. OpenAlex counts 107 citations as of 17 November 2026, which puts it in the top 1% of works published in 2026 by citations. It is open access under a CC BY licence: the full text is free to read at Nature Medicine.
- Citations
- 107
- Most cited in
- 2026 (106)
- Field-weighted impact
- 110.33
- References
- 14
- Authors
- 16
Abstract
Specialized clinical artificial intelligence (AI) tools are entering medical practice despite scarce independent evaluation. We quantitatively evaluate two clinical AI tools, OpenEvidence and UpToDate Expert AI, built on large language models (LLMs) against three frontier LLMs: GPT-5.2, Gemini 3.1 Pro and Claude Opus 4.6. Our evaluation has three stages: (1) 500 MedQA questions testing medical knowledge, (2) 500 HealthBench items measuring alignment with clinicians and (3) the real clinical queries (RCQ) benchmark, built from 100 de-identified queries from physicians to a general-purpose language model in a live clinical environment. For the RCQ benchmark, 12 US clinicians performed randomized, blinded review of model outputs, producing 1,800 model-question annotations. Frontier LLMs outperformed clinical AI tools in all three evaluations. Clinical AI tools performed comparably to auto-enabled Google Search AI Overview on the RCQ. These findings highlight the need for independent, real-world evaluation of AI tools before they enter clinical settings.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Vishwanath, K., Alyakin, A., Ghosh, M., Hage, A., Neifert, S. N., Orillac, C. M., Mandelberg, N. J., Khan, H. A., Lee, J. V., Yao, J. J., Small, W. R., Varma, A., Hewitt, D. B., Aphinyanaphongs, Y., Alber, D. A., & Oermann, E. K. (2026). General-purpose large language models outperform specialized clinical AI tools on medical benchmarks. Nature Medicine, 32(7), 2405–2409. https://doi.org/10.1038/s41591-026-04431-5
MLA 9
Vishwanath, Krithik, et al. “General-purpose large language models outperform specialized clinical AI tools on medical benchmarks.” Nature Medicine, vol. 32, no. 7, 2026, pp. 2405–2409. https://doi.org/10.1038/s41591-026-04431-5.
Chicago
Vishwanath, Krithik, Anton Alyakin, Mrigayu Ghosh, Ali Hage, Sean N. Neifert, Cordelia Marcela Orillac, Nataniel J. Mandelberg, et al. 2026. “General-purpose large language models outperform specialized clinical AI tools on medical benchmarks.” Nature Medicine 32 (7): 2405–2409. https://doi.org/10.1038/s41591-026-04431-5.
Harvard
Vishwanath, K. et al. (2026) ‘General-purpose large language models outperform specialized clinical AI tools on medical benchmarks’, Nature Medicine, 32(7), pp. 2405–2409. doi:10.1038/s41591-026-04431-5.
IEEE
K. Vishwanath et al., “General-purpose large language models outperform specialized clinical AI tools on medical benchmarks,” Nature Medicine, vol. 32, no. 7, pp. 2405–2409, 2026, doi: 10.1038/s41591-026-04431-5.
BibTeX
@article{vishwanath2026general,
title = {{General-purpose large language models outperform specialized clinical AI tools on medical benchmarks}},
author = {Vishwanath, Krithik and Alyakin, Anton and Ghosh, Mrigayu and Hage, Ali and Neifert, Sean N. and Orillac, Cordelia Marcela and Mandelberg, Nataniel J. and Khan, Hammad A. and Lee, Jin Vivian and Yao, Jie J. and Small, William Robert and Varma, Aakaash and Hewitt, Daniel Brock and Aphinyanaphongs, Yindalon and Alber, Daniel Alexander and Oermann, Eric Karl},
journal = {Nature Medicine},
year = {2026},
volume = {32},
number = {7},
pages = {2405--2409},
doi = {10.1038/s41591-026-04431-5},
url = {https://doi.org/10.1038/s41591-026-04431-5}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- large language models
- medical question answering
- MedQA
- OpenEvidence
- GPT-5.2
- Gemini
- Claude Opus
- clinical decision support
Work with this paper
Most-cited papers in Medicine
- Yaşlı Kadınlarda Üreme SağlığıDemet Kokanalı, Yaprak Engin Üstün · Jinekoloji-Obstetrik ve Neonatoloji Tıp Dergisi · 2019 · 169,630 citations
- Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 CountriesHyuna Sung, Adalberto M. Filho, Mathieu Laversanne and 5 more · CA A Cancer Journal for Clinicians · 2021 · 119,885 citations
- “Mini-mental state”Marshal F. Folstein, Susan E. Folstein, Paul R. McHugh · Journal of Psychiatric Research · 1975 · 90,424 citations
- Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countriesFreddie Bray, Jacques Ferlay, Isabelle Soerjomataram and 3 more · CA A Cancer Journal for Clinicians · 2018 · 87,396 citations
- Hallmarks of Cancer: The Next GenerationDouglas Hanahan, Robert A. Weinberg · Cell · 2011 · 69,758 citations
Questions about this paper
How do I cite this paper in APA style?
Vishwanath, K., Alyakin, A., Ghosh, M., Hage, A., Neifert, S. N., Orillac, C. M., Mandelberg, N. J., Khan, H. A., Lee, J. V., Yao, J. J., Small, W. R., Varma, A., Hewitt, D. B., Aphinyanaphongs, Y., Alber, D. A., & Oermann, E. K. (2026). General-purpose large language models outperform specialized clinical AI tools on medical benchmarks. Nature Medicine, 32(7), 2405–2409. https://doi.org/10.1038/s41591-026-04431-5
How many times has this paper been cited?
107 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Nature Medicine: sign in and open the PDF from this page.
Who wrote this paper?
Krithik Vishwanath, Anton Alyakin, Mrigayu Ghosh, Ali Hage, Sean N. Neifert, Cordelia Marcela Orillac, Nataniel J. Mandelberg, Hammad A. Khan, Jin Vivian Lee, Jie J. Yao, William Robert Small, Aakaash Varma and 4 more.
What is the DOI of this paper?
10.1038/s41591-026-04431-5, permanently at https://doi.org/10.1038/s41591-026-04431-5. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.