Preprint · 2026 · Open access
Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops
arXiv (Cornell University) · Published 12 June 2026 · DOI 10.48550/arxiv.2606.14149
In brief
“Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops” is a 2026 preprint by Muhammad Osama and 4 co-authors, published in arXiv (Cornell University). OpenAlex counts 3 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 3
- Most cited in
- 2026 (3)
- References
- 22
- Authors
- 5
Abstract
Large Language Models (LLMs) are increasingly deployed in healthcare settings, yet their tendency to hallucinate poses risks when clinical decisions are involved. This study examine whether LLMs recommend recently banned or withdrawn pharmaceuticals when answering clinical questions and tests an agent-based method for reducing such errors. We developed a five-agent "Trust but Verify" system using a single LLM backbone. To measure regulatory knowledge obsolescence, we created an adversarial dataset of 103 clinical MCQs where historically correct answers now refer to banned substances. This scale ensures statistical significance across various therapeutic classes. We evaluated three open-access model families (GPT-OSS, Llama-3, Falcon-3) under vanilla and agentic conditions. Performance was measured via pointwise score, label accuracy, Hallucination Error Rate (HER), and Component Fidelity (CF) score. We also observed clinical safety regression in proprietary models. In default configurations, all models showed high hallucination rates, consistently selecting banned drugs that matched training data patterns. Our proposed agentic architecture reduced HER by approximately 53% across models. Pointwise scores shifted from -0.25 (unsafe recommendation) toward 0.0 (appropriate refusal). The safety audit intercepted dangerous outputs even when models' parametric knowledge favored the banned substance. The proposed multi-agent framework offers a model-agnostic method for enforcing regulatory compliance that prioritizes patient safety over fluent text generation. Our work demonstrates a practical approach for deploying autonomous AI systems in safety-critical healthcare settings. It shows how real-time regulatory data can be integrated into LLM pipelines to support clinical decision-making.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Osama, M., Amjad, M., Mustansar, Z., Shaukat, A., & Khan, M. (2026). Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2606.14149
MLA 9
Osama, Muhammad, et al. “Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2606.14149.
Chicago
Osama, Muhammad, Maheera Amjad, Zartasha Mustansar, Arslan Shaukat, and Maham Khan. 2026. “Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2606.14149.
Harvard
Osama, M. et al. (2026) ‘Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops’, arXiv (Cornell University). doi:10.48550/arxiv.2606.14149.
IEEE
M. Osama, M. Amjad, Z. Mustansar, A. Shaukat, and M. Khan, “Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2606.14149.
BibTeX
@misc{osama2026trust,
title = {{Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops}},
author = {Osama, Muhammad and Amjad, Maheera and Mustansar, Zartasha and Shaukat, Arslan and Khan, Maham},
howpublished = {arXiv (Cornell University)},
year = {2026},
doi = {10.48550/arxiv.2606.14149},
url = {https://doi.org/10.48550/arxiv.2606.14149}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- large language models in healthcare
- multi-agent systems
- clinical decision support
- safety-critical AI
Work with this paper
Most-cited papers in Medicine
- Yaşlı Kadınlarda Üreme SağlığıDemet Kokanalı, Yaprak Engin Üstün · Jinekoloji-Obstetrik ve Neonatoloji Tıp Dergisi · 2019 · 169,630 citations
- Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 CountriesHyuna Sung, Adalberto M. Filho, Mathieu Laversanne and 5 more · CA A Cancer Journal for Clinicians · 2021 · 119,885 citations
- “Mini-mental state”Marshal F. Folstein, Susan E. Folstein, Paul R. McHugh · Journal of Psychiatric Research · 1975 · 90,424 citations
- Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countriesFreddie Bray, Jacques Ferlay, Isabelle Soerjomataram and 3 more · CA A Cancer Journal for Clinicians · 2018 · 87,396 citations
- Hallmarks of Cancer: The Next GenerationDouglas Hanahan, Robert A. Weinberg · Cell · 2011 · 69,758 citations
Questions about this paper
How do I cite this paper in APA style?
Osama, M., Amjad, M., Mustansar, Z., Shaukat, A., & Khan, M. (2026). Trust but Verify: Mitigating Medical Hallucinations via Post-Hoc Adversarial Auditing and Multi-Agent Feedback Loops. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2606.14149
How many times has this paper been cited?
3 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Muhammad Osama, Maheera Amjad, Zartasha Mustansar, Arslan Shaukat, Maham Khan.
What is the DOI of this paper?
10.48550/arxiv.2606.14149, permanently at https://doi.org/10.48550/arxiv.2606.14149. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.