Preprint · 2026 · Open access
A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
arXiv (Cornell University) · Published 1 October 2026 · DOI 10.48550/arxiv.2610.01938
In brief
“A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined” is a 2026 preprint by Zhangshu Joshua Jiang and 2 co-authors, published in arXiv (Cornell University). OpenAlex counts 0 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 0
- References
- 38
- Authors
- 3
Abstract
Exam-style accuracy does not establish whether large language models (LLMs) reason well over clinical records. We define clinical reasoning as integrating and updating evidence across time and sources to form, revise and justify a patient's problem representation and a defensible plan. This structured narrative review maps three literatures: medical education assessment instruments, clinical LLM benchmarks published from 2023 onwards, and general-domain methods for evaluating long-form generation. We examine six dimensions: problem representation, temporal synthesis, differential and management reasoning, counterfactual reasoning, calibrated uncertainty, and reasoning faithfulness. Preprints are included and flagged. No single instrument covers all six dimensions. Problem representation and differential or management reasoning are reasonably covered, although reliability varies by instrument and setting. TIMER-Eval targets temporal synthesis, and ER-Reason assesses sequential diagnostic belief updating. Dedicated uncertainty and counterfactual evaluations are emerging, but their applicability to longitudinal free-text reasoning remains limited. Factual completeness is well theorised in general-domain evaluation, with early clinical evidence of important omissions. Faithfulness remains the weakest dimension, with one identified clinical causal-ablation study on multiple-choice questions. Existing tools should be combined through binary rubric items, separate completeness and correctness scores, case-specific importance weighting with non-compensable safety caps, temporal order-consistency checks, and chance-corrected reliability reporting. Further design work is needed for calibrated uncertainty, counterfactual reasoning and faithfulness over longitudinal free-text records. This review provides a design rationale, not a validated instrument.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Jiang, Z. J., Ibrahim, Z. M., & Teo, J. (2026). A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2610.01938
MLA 9
Jiang, Zhangshu Joshua, et al. “A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2610.01938.
Chicago
Jiang, Zhangshu Joshua, Zina M. Ibrahim, and James Teo. 2026. “A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2610.01938.
Harvard
Jiang, Z. J., Ibrahim, Z. M. and Teo, J. (2026) ‘A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined’, arXiv (Cornell University). doi:10.48550/arxiv.2610.01938.
IEEE
Z. J. Jiang, Z. M. Ibrahim, and J. Teo, “A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2610.01938.
BibTeX
@misc{jiang2026rubric,
title = {{A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined}},
author = {Jiang, Zhangshu Joshua and Ibrahim, Zina M. and Teo, James},
howpublished = {arXiv (Cornell University)},
year = {2026},
doi = {10.48550/arxiv.2610.01938},
url = {https://doi.org/10.48550/arxiv.2610.01938}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- clinical reasoning
- large language models
- counterfactual reasoning
- faithfulness
- problem representation
- medical records
- medical education assessment
Work with this paper
Most-cited papers in Medicine
- Yaşlı Kadınlarda Üreme SağlığıDemet Kokanalı, Yaprak Engin Üstün · Jinekoloji-Obstetrik ve Neonatoloji Tıp Dergisi · 2019 · 169,630 citations
- Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 CountriesHyuna Sung, Adalberto M. Filho, Mathieu Laversanne and 5 more · CA A Cancer Journal for Clinicians · 2021 · 119,885 citations
- “Mini-mental state”Marshal F. Folstein, Susan E. Folstein, Paul R. McHugh · Journal of Psychiatric Research · 1975 · 90,424 citations
- Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countriesFreddie Bray, Jacques Ferlay, Isabelle Soerjomataram and 3 more · CA A Cancer Journal for Clinicians · 2018 · 87,396 citations
- Hallmarks of Cancer: The Next GenerationDouglas Hanahan, Robert A. Weinberg · Cell · 2011 · 69,758 citations
Questions about this paper
How do I cite this paper in APA style?
Jiang, Z. J., Ibrahim, Z. M., & Teo, J. (2026). A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2610.01938
How many times has this paper been cited?
0 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Zhangshu Joshua Jiang, Zina M. Ibrahim, James Teo.
What is the DOI of this paper?
10.48550/arxiv.2610.01938, permanently at https://doi.org/10.48550/arxiv.2610.01938. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.