Review article · 2026 · Open access

Artificial Authority: The Promise and Perils of LLM Judges in Healthcare

Ariana Genovese, Lars Hegstrom, Srinivasagam Prabha, Cesar Abraham Gomez-Cabello, Syed Ali Haider, Bernardo Gabriele Collaço, Nadia G. Wood, Antonio J. Forte

Bioengineering, 13(1), 108 · Published 16 January 2026 · DOI 10.3390/bioengineering13010108

Open PDFView at the publisherCite this paper

In brief

“Artificial Authority: The Promise and Perils of LLM Judges in Healthcare” is a 2026 review article by Ariana Genovese and 7 co-authors, published in Bioengineering. OpenAlex counts 15 citations as of 19 November 2026, which puts it in the top 1% of works published in 2026 by citations. It is open access under a CC BY licence: the full text is free to read at Bioengineering.

Citations
15
Most cited in
2026 (15)
Field-weighted impact
5.44
References
38
Authors
8

Abstract

BACKGROUND: Large language models (LLMs) are increasingly integrated into clinical documentation, decision support, and patient-facing applications across healthcare, including plastic and reconstructive surgery. Yet, their evaluation remains bottlenecked by costly, time-consuming human review. This has given rise to LLM-as-a-judge, in which LLMs are used to evaluate the outputs of other AI systems. METHODS: This review examines LLM-as-a-judge in healthcare with particular attention to judging architectures, validation strategies, and emerging applications. A narrative review of the literature was conducted, synthesizing LLM judge methodologies as well as judging paradigms, including those applied to clinical documentation, medical question-answering systems, and clinical conversation assessment. RESULTS: Across tasks, LLM judges align most closely with clinicians on objective criteria (e.g., factuality, grammaticality, internal consistency), benefit from structured evaluation and chain-of-thought prompting, and can approach or exceed inter-clinician agreement, but remain limited for subjective or affective judgments and by dataset quality and task specificity. CONCLUSIONS: The literature indicates that LLM judges can enable efficient, standardized evaluation in controlled settings; however, their appropriate role remains supportive rather than substitutive, and their performance may not generalize to complex plastic surgery environments. Their safe use depends on rigorous human oversight and explicit governance structures.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Genovese, A., Hegstrom, L., Prabha, S., Gomez-Cabello, C. A., Haider, S. A., Collaço, B. G., Wood, N. G., & Forte, A. J. (2026). Artificial Authority: The Promise and Perils of LLM Judges in Healthcare. Bioengineering, 13(1), 108. https://doi.org/10.3390/bioengineering13010108

MLA 9

Genovese, Ariana, et al. “Artificial Authority: The Promise and Perils of LLM Judges in Healthcare.” Bioengineering, vol. 13, no. 1, 2026, p. 108. https://doi.org/10.3390/bioengineering13010108.

Chicago

Genovese, Ariana, Lars Hegstrom, Srinivasagam Prabha, Cesar Abraham Gomez-Cabello, Syed Ali Haider, Bernardo Gabriele Collaço, Nadia G. Wood, and Antonio J. Forte. 2026. “Artificial Authority: The Promise and Perils of LLM Judges in Healthcare.” Bioengineering 13 (1): 108. https://doi.org/10.3390/bioengineering13010108.

Harvard

Genovese, A. et al. (2026) ‘Artificial Authority: The Promise and Perils of LLM Judges in Healthcare’, Bioengineering, 13(1), p. 108. doi:10.3390/bioengineering13010108.

IEEE

A. Genovese et al., “Artificial Authority: The Promise and Perils of LLM Judges in Healthcare,” Bioengineering, vol. 13, no. 1, p. 108, 2026, doi: 10.3390/bioengineering13010108.

BibTeX

@article{genovese2026artificial,
  title     = {{Artificial Authority: The Promise and Perils of LLM Judges in Healthcare}},
  author    = {Genovese, Ariana and Hegstrom, Lars and Prabha, Srinivasagam and Gomez-Cabello, Cesar Abraham and Haider, Syed Ali and Collaço, Bernardo Gabriele and Wood, Nadia G. and Forte, Antonio J.},
  journal   = {Bioengineering},
  year      = {2026},
  volume    = {13},
  number    = {1},
  pages     = {108},
  doi       = {10.3390/bioengineering13010108},
  url       = {https://doi.org/10.3390/bioengineering13010108}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education

  • large language models
  • LLM-as-a-judge
  • AI evaluation
  • clinical documentation
  • clinical decision support
  • medical question answering
  • plastic surgery
  • chain-of-thought prompting

Work with this paper

Most-cited papers in Medicine

All highly cited Medicine papers

Questions about this paper

How do I cite this paper in APA style?

Genovese, A., Hegstrom, L., Prabha, S., Gomez-Cabello, C. A., Haider, S. A., Collaço, B. G., Wood, N. G., & Forte, A. J. (2026). Artificial Authority: The Promise and Perils of LLM Judges in Healthcare. Bioengineering, 13(1), 108. https://doi.org/10.3390/bioengineering13010108

How many times has this paper been cited?

15 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Bioengineering: sign in and open the PDF from this page.

Who wrote this paper?

Ariana Genovese, Lars Hegstrom, Srinivasagam Prabha, Cesar Abraham Gomez-Cabello, Syed Ali Haider, Bernardo Gabriele Collaço, Nadia G. Wood, Antonio J. Forte.

What is the DOI of this paper?

10.3390/bioengineering13010108, permanently at https://doi.org/10.3390/bioengineering13010108. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.