Journal article · 2026 · Open access

MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment

Christopher Wong, Divy Kumar, Amiin Muse, Philip Wang, Finn Wintz, Sasha-ann East, Jason Lyou, Naveena V. K. Yanamala, Zach Wood-Doughty

JAMIA Open, 9(5), ooag143 · Published 18 July 2026 · DOI 10.1093/jamiaopen/ooag143

Open full textCite this paper

In brief

“MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment” is a 2026 journal article by Christopher Wong and 8 co-authors, published in JAMIA Open. OpenAlex counts 0 citations as of 19 November 2026. It is open access under a CC BY-NC licence: the full text is free to read at JAMIA Open.

Citations
0
References
32
Authors
9

Abstract

Objectives: To simultaneously evaluate and compare LLM performance in clinical reasoning with a minimally necessary criteria for clinical alignment with patients. Material and Methods: We adapt the MedNLI dataset to create MedNLI-Pain, a new benchmark task that evaluates if LLMs that make accurate clinical inferences about patients can also anticipate when those patients are in pain. MedNLI-Pain consists of patient vignettes originating from MedNLI that are then annotated with physician assessments of the described patient's pain. Results: All LLMs had higher agreement with physicians in MedNLI than in MedNLI-Pain and its counterfactually augmented subset. This trend persisted when accounting for the higher difficulty of MedNLI-Pain by normalizing to human performance. Large language model performance improved with more training data in MedNLI but not MedNLI-Pain. Discussion: Large language models may lack necessary capabilities (eg, recognizing patients' pain) for clinical alignment. Success at a clinical reasoning task, in this case medical natural language inference, may not entail commensurate ability to align with patients' interests. Conclusions: Further research on alignment with patients is needed because the implications may direct future research into either closed-ended or agentic applications.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Wong, C., Kumar, D., Muse, A., Wang, P., Wintz, F., East, S.-A., Lyou, J., Yanamala, N. V. K., & Wood-Doughty, Z. (2026). MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment. JAMIA Open, 9(5), ooag143. https://doi.org/10.1093/jamiaopen/ooag143

MLA 9

Wong, Christopher, et al. “MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment.” JAMIA Open, vol. 9, no. 5, 2026, p. ooag143. https://doi.org/10.1093/jamiaopen/ooag143.

Chicago

Wong, Christopher, Divy Kumar, Amiin Muse, Philip Wang, Finn Wintz, Sasha-ann East, Jason Lyou, Naveena V. K. Yanamala, and Zach Wood-Doughty. 2026. “MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment.” JAMIA Open 9 (5): ooag143. https://doi.org/10.1093/jamiaopen/ooag143.

Harvard

Wong, C. et al. (2026) ‘MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment’, JAMIA Open, 9(5), p. ooag143. doi:10.1093/jamiaopen/ooag143.

IEEE

C. Wong et al., “MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment,” JAMIA Open, vol. 9, no. 5, p. ooag143, 2026, doi: 10.1093/jamiaopen/ooag143.

BibTeX

@article{wong2026mednli,
  title     = {{MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment}},
  author    = {Wong, Christopher and Kumar, Divy and Muse, Amiin and Wang, Philip and Wintz, Finn and East, Sasha-ann and Lyou, Jason and Yanamala, Naveena V. K. and Wood-Doughty, Zach},
  journal   = {JAMIA Open},
  year      = {2026},
  volume    = {9},
  number    = {5},
  pages     = {ooag143},
  doi       = {10.1093/jamiaopen/ooag143},
  url       = {https://doi.org/10.1093/jamiaopen/ooag143}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education

  • large language models
  • clinical reasoning
  • benchmarking
  • patient-physician agreement
  • training data size

Work with this paper

Most-cited papers in Medicine

All highly cited Medicine papers

Questions about this paper

How do I cite this paper in APA style?

Wong, C., Kumar, D., Muse, A., Wang, P., Wintz, F., East, S.-A., Lyou, J., Yanamala, N. V. K., & Wood-Doughty, Z. (2026). MedNLI-Pain benchmarks a minimally necessary requirement for large language model alignment. JAMIA Open, 9(5), ooag143. https://doi.org/10.1093/jamiaopen/ooag143

How many times has this paper been cited?

0 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at JAMIA Open: sign in and open the full text from this page.

Who wrote this paper?

Christopher Wong, Divy Kumar, Amiin Muse, Philip Wang, Finn Wintz, Sasha-ann East, Jason Lyou, Naveena V. K. Yanamala, Zach Wood-Doughty.

What is the DOI of this paper?

10.1093/jamiaopen/ooag143, permanently at https://doi.org/10.1093/jamiaopen/ooag143. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.