Journal article · 2025 · Open access
A HIPAA-Aware Benchmark and Evaluation Harness for Clinical LLMs to Quantify Hallucination, Bias, and PHI Leakage
Well Testing Journal · Published 18 August 2025
In brief
“A HIPAA-Aware Benchmark and Evaluation Harness for Clinical LLMs to Quantify Hallucination, Bias, and PHI Leakage” is a 2025 journal article by Valentina Palama, published in Well Testing Journal. OpenAlex counts 1 citations as of 19 November 2026. It is open access under a CC BY-NC licence: the full text is free to read at Well Testing Journal.
- Citations
- 1
- Most cited in
- 2026 (1)
- Field-weighted impact
- 0.15
- Authors
- 1
Abstract
The growing use of large language models (LLMs) in clinical practice has brought up serious doubts about the reliability, fairness, and compliance with regulations. Model hallucinations may undermine clinical decision-making in a health care setting, algorithmic bias may create health inequities, and unintended disclosure of protected health information (PHI), may contravene privacy rules. Although more focus is given to clinical LLM assessment, current benchmarks pay much attention to overall performance and do not address these safety and compliance risks in a holistic manner in a HIPAA-conscious system. This paper suggests a uniform benchmark and assessment harness that is specifically rigorously developed in clinical LLMs to quantify hallucination, bias, and leakage of PHI systematically. The framework includes clinically-based exercises, de-identified and artificial data sets, and automated identification tools consistent with the HIPAA-related privacy categories. Through the combination of several dimensions of evaluation into a single and reproducible harness, the benchmark can allow comparative clinical LLM evaluation of safety, fairness and privacy measures. The findings demonstrate that there is a significant variability in model behavior, which indicates trade-offs between clinical capability and risk exposure. The work adds a useful evaluation system to help implement LLMs responsibly, regulate, and monitor their use in healthcare settings.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Palama, V. (2025). A HIPAA-Aware Benchmark and Evaluation Harness for Clinical LLMs to Quantify Hallucination, Bias, and PHI Leakage. Well Testing Journal. https://welltestingjournal.com/index.php/WT/article/download/274/224
MLA 9
Palama, Valentina. “A HIPAA-Aware Benchmark and Evaluation Harness for Clinical LLMs to Quantify Hallucination, Bias, and PHI Leakage.” Well Testing Journal, 2025. https://welltestingjournal.com/index.php/WT/article/download/274/224.
Chicago
Palama, Valentina. 2025. “A HIPAA-Aware Benchmark and Evaluation Harness for Clinical LLMs to Quantify Hallucination, Bias, and PHI Leakage.” Well Testing Journal. https://welltestingjournal.com/index.php/WT/article/download/274/224.
Harvard
Palama, V. (2025) ‘A HIPAA-Aware Benchmark and Evaluation Harness for Clinical LLMs to Quantify Hallucination, Bias, and PHI Leakage’, Well Testing Journal. Available at: https://welltestingjournal.com/index.php/WT/article/download/274/224.
IEEE
V. Palama, “A HIPAA-Aware Benchmark and Evaluation Harness for Clinical LLMs to Quantify Hallucination, Bias, and PHI Leakage,” Well Testing Journal, 2025.
BibTeX
@article{palama2025hipaa,
title = {{A HIPAA-Aware Benchmark and Evaluation Harness for Clinical LLMs to Quantify Hallucination, Bias, and PHI Leakage}},
author = {Palama, Valentina},
journal = {Well Testing Journal},
year = {2025},
url = {https://welltestingjournal.com/index.php/WT/article/download/274/224}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- clinical large language models
- HIPAA compliance
- model hallucination
- algorithmic bias
- benchmarking
- evaluation harness
- privacy
- fairness
Work with this paper
Most-cited papers in Medicine
- Yaşlı Kadınlarda Üreme SağlığıDemet Kokanalı, Yaprak Engin Üstün · Jinekoloji-Obstetrik ve Neonatoloji Tıp Dergisi · 2019 · 169,630 citations
- Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 CountriesHyuna Sung, Adalberto M. Filho, Mathieu Laversanne and 5 more · CA A Cancer Journal for Clinicians · 2021 · 119,885 citations
- “Mini-mental state”Marshal F. Folstein, Susan E. Folstein, Paul R. McHugh · Journal of Psychiatric Research · 1975 · 90,424 citations
- Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countriesFreddie Bray, Jacques Ferlay, Isabelle Soerjomataram and 3 more · CA A Cancer Journal for Clinicians · 2018 · 87,396 citations
- Hallmarks of Cancer: The Next GenerationDouglas Hanahan, Robert A. Weinberg · Cell · 2011 · 69,758 citations
Questions about this paper
How do I cite this paper in APA style?
Palama, V. (2025). A HIPAA-Aware Benchmark and Evaluation Harness for Clinical LLMs to Quantify Hallucination, Bias, and PHI Leakage. Well Testing Journal. https://welltestingjournal.com/index.php/WT/article/download/274/224
How many times has this paper been cited?
1 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Well Testing Journal: sign in and open the full text from this page.
Who wrote this paper?
Valentina Palama.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.