Journal article · 2026 · Open access

General-purpose large language models outperform specialized clinical AI tools on medical benchmarks

Krithik Vishwanath, Anton Alyakin, Mrigayu Ghosh, Ali Hage, Sean N. Neifert, Cordelia Marcela Orillac, Nataniel J. Mandelberg, Hammad A. Khan, Jin Vivian Lee, Jie J. Yao, William Robert Small, Aakaash Varma and 4 more

Nature Medicine, 32(7), 2405–2409 · Published 12 June 2026 · DOI 10.1038/s41591-026-04431-5

Open PDFView at the publisherCite this paper

In brief

“General-purpose large language models outperform specialized clinical AI tools on medical benchmarks” is a 2026 journal article by Krithik Vishwanath and 15 co-authors, published in Nature Medicine. OpenAlex counts 107 citations as of 17 November 2026, which puts it in the top 1% of works published in 2026 by citations. It is open access under a CC BY licence: the full text is free to read at Nature Medicine.

Citations
107
Most cited in
2026 (106)
Field-weighted impact
110.33
References
14
Authors
16

Abstract

Specialized clinical artificial intelligence (AI) tools are entering medical practice despite scarce independent evaluation. We quantitatively evaluate two clinical AI tools, OpenEvidence and UpToDate Expert AI, built on large language models (LLMs) against three frontier LLMs: GPT-5.2, Gemini 3.1 Pro and Claude Opus 4.6. Our evaluation has three stages: (1) 500 MedQA questions testing medical knowledge, (2) 500 HealthBench items measuring alignment with clinicians and (3) the real clinical queries (RCQ) benchmark, built from 100 de-identified queries from physicians to a general-purpose language model in a live clinical environment. For the RCQ benchmark, 12 US clinicians performed randomized, blinded review of model outputs, producing 1,800 model-question annotations. Frontier LLMs outperformed clinical AI tools in all three evaluations. Clinical AI tools performed comparably to auto-enabled Google Search AI Overview on the RCQ. These findings highlight the need for independent, real-world evaluation of AI tools before they enter clinical settings.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Vishwanath, K., Alyakin, A., Ghosh, M., Hage, A., Neifert, S. N., Orillac, C. M., Mandelberg, N. J., Khan, H. A., Lee, J. V., Yao, J. J., Small, W. R., Varma, A., Hewitt, D. B., Aphinyanaphongs, Y., Alber, D. A., & Oermann, E. K. (2026). General-purpose large language models outperform specialized clinical AI tools on medical benchmarks. Nature Medicine, 32(7), 2405–2409. https://doi.org/10.1038/s41591-026-04431-5

MLA 9

Vishwanath, Krithik, et al. “General-purpose large language models outperform specialized clinical AI tools on medical benchmarks.” Nature Medicine, vol. 32, no. 7, 2026, pp. 2405–2409. https://doi.org/10.1038/s41591-026-04431-5.

Chicago

Vishwanath, Krithik, Anton Alyakin, Mrigayu Ghosh, Ali Hage, Sean N. Neifert, Cordelia Marcela Orillac, Nataniel J. Mandelberg, et al. 2026. “General-purpose large language models outperform specialized clinical AI tools on medical benchmarks.” Nature Medicine 32 (7): 2405–2409. https://doi.org/10.1038/s41591-026-04431-5.

Harvard

Vishwanath, K. et al. (2026) ‘General-purpose large language models outperform specialized clinical AI tools on medical benchmarks’, Nature Medicine, 32(7), pp. 2405–2409. doi:10.1038/s41591-026-04431-5.

IEEE

K. Vishwanath et al., “General-purpose large language models outperform specialized clinical AI tools on medical benchmarks,” Nature Medicine, vol. 32, no. 7, pp. 2405–2409, 2026, doi: 10.1038/s41591-026-04431-5.

BibTeX

@article{vishwanath2026general,
  title     = {{General-purpose large language models outperform specialized clinical AI tools on medical benchmarks}},
  author    = {Vishwanath, Krithik and Alyakin, Anton and Ghosh, Mrigayu and Hage, Ali and Neifert, Sean N. and Orillac, Cordelia Marcela and Mandelberg, Nataniel J. and Khan, Hammad A. and Lee, Jin Vivian and Yao, Jie J. and Small, William Robert and Varma, Aakaash and Hewitt, Daniel Brock and Aphinyanaphongs, Yindalon and Alber, Daniel Alexander and Oermann, Eric Karl},
  journal   = {Nature Medicine},
  year      = {2026},
  volume    = {32},
  number    = {7},
  pages     = {2405--2409},
  doi       = {10.1038/s41591-026-04431-5},
  url       = {https://doi.org/10.1038/s41591-026-04431-5}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education

  • large language models
  • medical question answering
  • MedQA
  • OpenEvidence
  • GPT-5.2
  • Gemini
  • Claude Opus
  • clinical decision support

Work with this paper

Most-cited papers in Medicine

All highly cited Medicine papers

Questions about this paper

How do I cite this paper in APA style?

Vishwanath, K., Alyakin, A., Ghosh, M., Hage, A., Neifert, S. N., Orillac, C. M., Mandelberg, N. J., Khan, H. A., Lee, J. V., Yao, J. J., Small, W. R., Varma, A., Hewitt, D. B., Aphinyanaphongs, Y., Alber, D. A., & Oermann, E. K. (2026). General-purpose large language models outperform specialized clinical AI tools on medical benchmarks. Nature Medicine, 32(7), 2405–2409. https://doi.org/10.1038/s41591-026-04431-5

How many times has this paper been cited?

107 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Nature Medicine: sign in and open the PDF from this page.

Who wrote this paper?

Krithik Vishwanath, Anton Alyakin, Mrigayu Ghosh, Ali Hage, Sean N. Neifert, Cordelia Marcela Orillac, Nataniel J. Mandelberg, Hammad A. Khan, Jin Vivian Lee, Jie J. Yao, William Robert Small, Aakaash Varma and 4 more.

What is the DOI of this paper?

10.1038/s41591-026-04431-5, permanently at https://doi.org/10.1038/s41591-026-04431-5. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.