Journal article · 2026 · Open access

Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study

Hassan Tarek Hakam, Nicolas Vuillerme, Felix Muehlensiepen, Nikolai Ramadanov, Jonathan Lettner, Marko Ostojić, Roland Becker, Robert Prill

Journal of Medical Internet Research, 28, e91095 · Published 13 November 2026 · DOI 10.2196/91095

Open full textCite this paper

In brief

“Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study” is a 2026 journal article by Hassan Tarek Hakam and 7 co-authors, published in Journal of Medical Internet Research. OpenAlex counts 0 citations as of 18 November 2026. It is open access under a CC BY licence: the full text is free to read at Journal of Medical Internet Research.

Citations
0
References
111
Authors
8

Abstract

BACKGROUND: Large language models (LLMs) are increasingly used in academic writing and can generate linguistically polished but factually incorrect scientific content. Such misinformation may distort evidence appraisal and, if translated into clinical practice, compromise patient care. OBJECTIVE: This study aimed to estimate physicians' likelihood of misperceiving false information embedded in LLM-generated abstracts as true and reporting an intent to use that information in future practice. We also explored which features were associated with perceived credibility and intended clinical action. METHODS: We conducted a cross-sectional, vignette-based survey of physicians working in emergency medicine and trauma surgery across 4 hospitals in Germany. Each participant assessed 4 abstracts generated using GPT-5 Thinking mode, of which 2 contained intentionally false claims supported by plausible explanations and 2 contained accurate claims. The primary outcome was the proportion of physicians reporting an intent to use or act on recommendations contained in abstracts with false claims. Secondary outcomes included perceived accuracy, rejection of accurate claims, and ratings of precision, structure, and grammar. RESULTS: Twenty physicians provided 80 ratings of 4 abstracts. Overall, physicians reported that they would act on 68% (54/80) of abstracts. Using abstract truth status as the reference, the decision to act had a sensitivity of 78% (31/40), specificity of 43% (17/40), positive predictive value of 57% (31/54), and negative predictive value of 65% (17/26). The positive likelihood ratio was 1.35 (95% CI 0.98-1.85), the negative likelihood ratio was 0.53 (95% CI 0.27-1.04), and the diagnostic odds ratio was 2.55 (95% CI 0.96-6.73). True and false abstracts received similar ratings for perceived accuracy (mean 3.15, SD 0.89 vs mean 2.88, SD 1.04; P=.21), precision (mean 3.15, SD 0.89 vs mean 2.93, SD 1.00; P=.29), structure (mean 3.23, SD 0.77 vs mean 3.20, SD 0.88; P=.89), and grammar (mean 3.40, SD 0.59 vs mean 3.43, SD 0.68; P=.86). Abstracts that physicians intended to act on were rated substantially higher for perceived accuracy, precision, and structure than those they would not act on (all P<.001), whereas grammar ratings did not differ (P=.80). CONCLUSIONS: Physicians frequently reported an intent to act on LLM-generated abstracts containing false claims, and true and false abstracts were not reliably distinguished on perceived quality measures. Perceived accuracy, precision, and structure were strongly associated with intended action, whereas grammar was not. These findings highlight the risk that persuasive scientific presentation may increase the credibility and potential clinical influence of fabricated or inaccurate AI-generated evidence.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Hakam, H. T., Vuillerme, N., Muehlensiepen, F., Ramadanov, N., Lettner, J., Ostojić, M., Becker, R., & Prill, R. (2026). Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study. Journal of Medical Internet Research, 28, e91095. https://doi.org/10.2196/91095

MLA 9

Hakam, Hassan Tarek, et al. “Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study.” Journal of Medical Internet Research, vol. 28, 2026, p. e91095. https://doi.org/10.2196/91095.

Chicago

Hakam, Hassan Tarek, Nicolas Vuillerme, Felix Muehlensiepen, Nikolai Ramadanov, Jonathan Lettner, Marko Ostojić, Roland Becker, and Robert Prill. 2026. “Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study.” Journal of Medical Internet Research 28: e91095. https://doi.org/10.2196/91095.

Harvard

Hakam, H. T. et al. (2026) ‘Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study’, Journal of Medical Internet Research, 28, p. e91095. doi:10.2196/91095.

IEEE

H. T. Hakam et al., “Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study,” Journal of Medical Internet Research, vol. 28, p. e91095, 2026, doi: 10.2196/91095.

BibTeX

@article{hakam2026hallucinated,
  title     = {{Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study}},
  author    = {Hakam, Hassan Tarek and Vuillerme, Nicolas and Muehlensiepen, Felix and Ramadanov, Nikolai and Lettner, Jonathan and Ostojić, Marko and Becker, Roland and Prill, Robert},
  journal   = {Journal of Medical Internet Research},
  year      = {2026},
  volume    = {28},
  pages     = {e91095},
  doi       = {10.2196/91095},
  url       = {https://doi.org/10.2196/91095}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education

  • large language models
  • AI-generated abstracts
  • GPT-5
  • diagnostic accuracy
  • misinformation
  • emergency medicine
  • trauma surgery
  • perceived credibility

Work with this paper

Most-cited papers in Medicine

All highly cited Medicine papers

Questions about this paper

How do I cite this paper in APA style?

Hakam, H. T., Vuillerme, N., Muehlensiepen, F., Ramadanov, N., Lettner, J., Ostojić, M., Becker, R., & Prill, R. (2026). Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study. Journal of Medical Internet Research, 28, e91095. https://doi.org/10.2196/91095

How many times has this paper been cited?

0 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Journal of Medical Internet Research: sign in and open the full text from this page.

Who wrote this paper?

Hassan Tarek Hakam, Nicolas Vuillerme, Felix Muehlensiepen, Nikolai Ramadanov, Jonathan Lettner, Marko Ostojić, Roland Becker, Robert Prill.

What is the DOI of this paper?

10.2196/91095, permanently at https://doi.org/10.2196/91095. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.