Journal article · 2026 · Open access
MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment
JAMIA Open, 9(5), ooag143 · Published 18 July 2026 · DOI 10.1093/jamiaopen/ooag143
In brief
“MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment” is a 2026 journal article by Christopher Wong and 8 co-authors, published in JAMIA Open. OpenAlex counts 0 citations as of 19 November 2026. It is open access under a CC BY-NC licence: the full text is free to read at JAMIA Open.
- Citations
- 0
- References
- 32
- Authors
- 9
Abstract
Objectives: To simultaneously evaluate and compare LLM performance in clinical reasoning with a minimally necessary criteria for clinical alignment with patients. Material and Methods: We adapt the MedNLI dataset to create MedNLI-Pain, a new benchmark task that evaluates if LLMs that make accurate clinical inferences about patients can also anticipate when those patients are in pain. MedNLI-Pain consists of patient vignettes originating from MedNLI that are then annotated with physician assessments of the described patient's pain. Results: All LLMs had higher agreement with physicians in MedNLI than in MedNLI-Pain and its counterfactually augmented subset. This trend persisted when accounting for the higher difficulty of MedNLI-Pain by normalizing to human performance. Large language model performance improved with more training data in MedNLI but not MedNLI-Pain. Discussion: Large language models may lack necessary capabilities (eg, recognizing patients' pain) for clinical alignment. Success at a clinical reasoning task, in this case medical natural language inference, may not entail commensurate ability to align with patients' interests. Conclusions: Further research on alignment with patients is needed because the implications may direct future research into either closed-ended or agentic applications.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Wong, C., Kumar, D., Muse, A., Wang, P., Wintz, F., East, S.-A., Lyou, J., Yanamala, N. V. K., & Wood-Doughty, Z. (2026). MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment. JAMIA Open, 9(5), ooag143. https://doi.org/10.1093/jamiaopen/ooag143
MLA 9
Wong, Christopher, et al. “MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment.” JAMIA Open, vol. 9, no. 5, 2026, p. ooag143. https://doi.org/10.1093/jamiaopen/ooag143.
Chicago
Wong, Christopher, Divy Kumar, Amiin Muse, Philip Wang, Finn Wintz, Sasha-ann East, Jason Lyou, Naveena V. K. Yanamala, and Zach Wood-Doughty. 2026. “MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment.” JAMIA Open 9 (5): ooag143. https://doi.org/10.1093/jamiaopen/ooag143.
Harvard
Wong, C. et al. (2026) ‘MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment’, JAMIA Open, 9(5), p. ooag143. doi:10.1093/jamiaopen/ooag143.
IEEE
C. Wong et al., “MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment,” JAMIA Open, vol. 9, no. 5, p. ooag143, 2026, doi: 10.1093/jamiaopen/ooag143.
BibTeX
@article{wong2026mednli,
title = {{MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment}},
author = {Wong, Christopher and Kumar, Divy and Muse, Amiin and Wang, Philip and Wintz, Finn and East, Sasha-ann and Lyou, Jason and Yanamala, Naveena V. K. and Wood-Doughty, Zach},
journal = {JAMIA Open},
year = {2026},
volume = {9},
number = {5},
pages = {ooag143},
doi = {10.1093/jamiaopen/ooag143},
url = {https://doi.org/10.1093/jamiaopen/ooag143}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- large language models
- clinical reasoning
- benchmarking
- patient-physician agreement
- training data size
Work with this paper
Most-cited papers in Medicine
- Yaşlı Kadınlarda Üreme SağlığıDemet Kokanalı, Yaprak Engin Üstün · Jinekoloji-Obstetrik ve Neonatoloji Tıp Dergisi · 2019 · 169,630 citations
- Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 CountriesHyuna Sung, Adalberto M. Filho, Mathieu Laversanne and 5 more · CA A Cancer Journal for Clinicians · 2021 · 119,885 citations
- “Mini-mental state”Marshal F. Folstein, Susan E. Folstein, Paul R. McHugh · Journal of Psychiatric Research · 1975 · 90,424 citations
- Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countriesFreddie Bray, Jacques Ferlay, Isabelle Soerjomataram and 3 more · CA A Cancer Journal for Clinicians · 2018 · 87,396 citations
- Hallmarks of Cancer: The Next GenerationDouglas Hanahan, Robert A. Weinberg · Cell · 2011 · 69,758 citations
Questions about this paper
How do I cite this paper in APA style?
Wong, C., Kumar, D., Muse, A., Wang, P., Wintz, F., East, S.-A., Lyou, J., Yanamala, N. V. K., & Wood-Doughty, Z. (2026). MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment. JAMIA Open, 9(5), ooag143. https://doi.org/10.1093/jamiaopen/ooag143
How many times has this paper been cited?
0 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at JAMIA Open: sign in and open the full text from this page.
Who wrote this paper?
Christopher Wong, Divy Kumar, Amiin Muse, Philip Wang, Finn Wintz, Sasha-ann East, Jason Lyou, Naveena V. K. Yanamala, Zach Wood-Doughty.
What is the DOI of this paper?
10.1093/jamiaopen/ooag143, permanently at https://doi.org/10.1093/jamiaopen/ooag143. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.