Conference paper · 2026 · Open access

MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks

Jianrong Lu, Yixue Li, Xingyun Zheng, Minghui Yang, Jian Kang Wang, Ping Wang, Yechao Zhang

Proceedings of the AAAI Conference on Artificial Intelligence, 40(45), 38971–38978 · Published 14 March 2026 · DOI 10.1609/aaai.v40i45.41243

Open PDFView at the publisherCite this paper

In brief

“MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks” is a 2026 conference paper by Jianrong Lu and 6 co-authors, published in Proceedings of the AAAI Conference on Artificial Intelligence. OpenAlex counts 7 citations as of 19 November 2026, which puts it in the top 2% of works published in 2026 by citations. It is open access: the full text is free to read at Proceedings of the AAAI Conference on Artificial Intelligence.

Citations
7
Most cited in
2026 (7)
Field-weighted impact
11.15
References
23
Authors
7

Abstract

The integration of Large Language Models (LLMs) into clinical applications presents transformative potential but is undermined by the critical risk of hallucination, the generation of plausible but factually incorrect information. Such failures pose a direct threat to patient safety and the integrity of clinical decision-making. To address this challenge, we introduce MHB, a novel and comprehensive benchmark framework designed to evaluate LLM reliability in two complex, high-stakes clinical contexts: multi-turn medical dialogues and clinical case report analysis. The core of our contribution is a systematic methodology for generating adversarial test cases by injecting ``hallucination traps" into realistic medical data, guided by a fine-grained taxonomy of clinical errors. MHB, comprising 4,695 samples and 20,288 evaluation rubrics, underwent a rigorous, two-stage validation by a panel of 60 licensed physicians from top-tier hospitals, ensuring high clinical realism and consistency. This comprehensive assessment of leading LLMs revealed significant, clinically relevant shortcomings across the board. Even the best-performing model, Claude-4-Sonnet, exhibited a hallucination rate of 29.1%, with some open-source models exceeding 57.0%. All models struggled with specific traps, like fabricated medical data or non-existent guidelines, highlighting prevalent systemic weaknesses.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Lu, J., Li, Y., Zheng, X., Yang, M., Wang, J. K., Wang, P., & Zhang, Y. (2026). MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 40(45), 38971–38978. https://doi.org/10.1609/aaai.v40i45.41243

MLA 9

Lu, Jianrong, et al. “MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks.” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 45, 2026, pp. 38971–38978. https://doi.org/10.1609/aaai.v40i45.41243.

Chicago

Lu, Jianrong, Yixue Li, Xingyun Zheng, Minghui Yang, Jian Kang Wang, Ping Wang, and Yechao Zhang. 2026. “MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks.” Proceedings of the AAAI Conference on Artificial Intelligence 40 (45): 38971–38978. https://doi.org/10.1609/aaai.v40i45.41243.

Harvard

Lu, J. et al. (2026) ‘MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks’, Proceedings of the AAAI Conference on Artificial Intelligence, 40(45), pp. 38971–38978. doi:10.1609/aaai.v40i45.41243.

IEEE

J. Lu et al., “MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 45, pp. 38971–38978, 2026, doi: 10.1609/aaai.v40i45.41243.

BibTeX

@inproceedings{lu2026medical,
  title     = {{MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks}},
  author    = {Lu, Jianrong and Li, Yixue and Zheng, Xingyun and Yang, Minghui and Wang, Jian Kang and Wang, Ping and Zhang, Yechao},
  booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year      = {2026},
  volume    = {40},
  number    = {45},
  pages     = {38971--38978},
  doi       = {10.1609/aaai.v40i45.41243},
  url       = {https://doi.org/10.1609/aaai.v40i45.41243}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education

  • hallucinations
  • large language models
  • clinical decision support
  • patient safety

Work with this paper

Most-cited papers in Medicine

All highly cited Medicine papers

Questions about this paper

How do I cite this paper in APA style?

Lu, J., Li, Y., Zheng, X., Yang, M., Wang, J. K., Wang, P., & Zhang, Y. (2026). MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 40(45), 38971–38978. https://doi.org/10.1609/aaai.v40i45.41243

How many times has this paper been cited?

7 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Proceedings of the AAAI Conference on Artificial Intelligence: sign in and open the PDF from this page.

Who wrote this paper?

Jianrong Lu, Yixue Li, Xingyun Zheng, Minghui Yang, Jian Kang Wang, Ping Wang, Yechao Zhang.

What is the DOI of this paper?

10.1609/aaai.v40i45.41243, permanently at https://doi.org/10.1609/aaai.v40i45.41243. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.