Review article · 2026 · Open access

LLM-assisted systematic review of large language models in clinical medicine

Sully F. Chen, Anton Alyakin, Andreas Seas, Eunice Yang, Joanne J. Choi, Jin Vivian Lee, Amelia L. Chen, Pranav I. Warman, Rochelle Bitolas, Robert J. Steele, Daniel Alexander Alber, Eric Karl Oermann

Nature Medicine, 32(3), 1152–1159 · Published 1 March 2026 · DOI 10.1038/s41591-026-04229-5

Open PDFView at the publisherCite this paper

In brief

“LLM-assisted systematic review of large language models in clinical medicine” is a 2026 review article by Sully F. Chen and 11 co-authors, published in Nature Medicine. OpenAlex counts 141 citations as of 17 November 2026. It is open access under a CC BY-NC-ND licence: the full text is free to read at Nature Medicine.

Citations
141
Most cited in
2026 (140)
Field-weighted impact
45.58
References
20
Authors
12

Abstract

Clinical evaluations of large language models (LLMs) have rapidly expanded since 2022, yet their evidence base remains opaque. The overwhelming volume of studies creates challenges for manual curation and review. However, LLMs themselves offer the scalability and capability to evaluate the ever-growing evidence base. This LLM-assisted review identified 4,609 peer-reviewed studies in clinical medicine between January 2022 and September 2025, equating to roughly 3.2 papers per day. Only 1,048 studies used real-world patient data and of these only 19 were prospective randomized trials; most addressed simulated scenarios (n = 1,857) or exam-style tasks (n = 1,704). ChatGPT and related OpenAI models constitute 65.7% of evaluated models, with Gemini/Bard a distant second constituting 13.1% of evaluated models. Patient-facing communication and education comprised 17% of tasks, followed by knowledge retrieval, and education and assessment simulation. Across 1,046 head-to-head comparisons, LLMs outperformed humans in 33% of comparisons, with a strong dependency on task realism and level of training. At least 25% of studies had sample sizes less than 30. Despite the growth of LLMs in medicine, rigorous, patient-centered evidence remains scarce, underscoring the need for larger prospective trials before clinical adoption.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Chen, S. F., Alyakin, A., Seas, A., Yang, E., Choi, J. J., Lee, J. V., Chen, A. L., Warman, P. I., Bitolas, R., Steele, R. J., Alber, D. A., & Oermann, E. K. (2026). LLM-assisted systematic review of large language models in clinical medicine. Nature Medicine, 32(3), 1152–1159. https://doi.org/10.1038/s41591-026-04229-5

MLA 9

Chen, Sully F., et al. “LLM-assisted systematic review of large language models in clinical medicine.” Nature Medicine, vol. 32, no. 3, 2026, pp. 1152–1159. https://doi.org/10.1038/s41591-026-04229-5.

Chicago

Chen, Sully F., Anton Alyakin, Andreas Seas, Eunice Yang, Joanne J. Choi, Jin Vivian Lee, Amelia L. Chen, et al. 2026. “LLM-assisted systematic review of large language models in clinical medicine.” Nature Medicine 32 (3): 1152–1159. https://doi.org/10.1038/s41591-026-04229-5.

Harvard

Chen, S. F. et al. (2026) ‘LLM-assisted systematic review of large language models in clinical medicine’, Nature Medicine, 32(3), pp. 1152–1159. doi:10.1038/s41591-026-04229-5.

IEEE

S. F. Chen et al., “LLM-assisted systematic review of large language models in clinical medicine,” Nature Medicine, vol. 32, no. 3, pp. 1152–1159, 2026, doi: 10.1038/s41591-026-04229-5.

BibTeX

@article{chen2026assisted,
  title     = {{LLM-assisted systematic review of large language models in clinical medicine}},
  author    = {Chen, Sully F. and Alyakin, Anton and Seas, Andreas and Yang, Eunice and Choi, Joanne J. and Lee, Jin Vivian and Chen, Amelia L. and Warman, Pranav I. and Bitolas, Rochelle and Steele, Robert J. and Alber, Daniel Alexander and Oermann, Eric Karl},
  journal   = {Nature Medicine},
  year      = {2026},
  volume    = {32},
  number    = {3},
  pages     = {1152--1159},
  doi       = {10.1038/s41591-026-04229-5},
  url       = {https://doi.org/10.1038/s41591-026-04229-5}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education

  • large language models
  • clinical medicine
  • evidence base
  • ChatGPT
  • Gemini
  • knowledge retrieval
  • head-to-head comparison
  • real-world patient data

Work with this paper

Most-cited papers in Medicine

All highly cited Medicine papers

Questions about this paper

How do I cite this paper in APA style?

Chen, S. F., Alyakin, A., Seas, A., Yang, E., Choi, J. J., Lee, J. V., Chen, A. L., Warman, P. I., Bitolas, R., Steele, R. J., Alber, D. A., & Oermann, E. K. (2026). LLM-assisted systematic review of large language models in clinical medicine. Nature Medicine, 32(3), 1152–1159. https://doi.org/10.1038/s41591-026-04229-5

How many times has this paper been cited?

141 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Nature Medicine: sign in and open the PDF from this page.

Who wrote this paper?

Sully F. Chen, Anton Alyakin, Andreas Seas, Eunice Yang, Joanne J. Choi, Jin Vivian Lee, Amelia L. Chen, Pranav I. Warman, Rochelle Bitolas, Robert J. Steele, Daniel Alexander Alber, Eric Karl Oermann.

What is the DOI of this paper?

10.1038/s41591-026-04229-5, permanently at https://doi.org/10.1038/s41591-026-04229-5. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.