Journal article · 2026 · Open access
Model-dependent prompt effects challenge reliable deployment of medical LLMs in traditional Chinese medicine
npj Digital Medicine · Published 13 October 2026 · DOI 10.1038/s41746-026-03397-5
In brief
“Model-dependent prompt effects challenge reliable deployment of medical LLMs in traditional Chinese medicine” is a 2026 journal article by Liang-Wei Tseng and 4 co-authors, published in npj Digital Medicine. OpenAlex counts 1 citations as of 17 November 2026, which puts it in the top 9% of works published in 2026 by citations. It is open access under a CC BY-NC-ND licence: the full text is free to read at npj Digital Medicine.
- Citations
- 1
- Most cited in
- 2026 (1)
- Field-weighted impact
- 1.20
- References
- 42
- Authors
- 5
Abstract
Although contemporary large language models (LLMs) have achieved satisfactory results in Western medical knowledge, their performance on traditional Chinese medicine (TCM) remains underexplored, and the determinants of performance variation are poorly understood. We evaluated 16 LLMs under seven prompt strategies on 475 questions from the 2025 Taiwan national TCM licensing examination, a public benchmark of TCM knowledge. Without prompt engineering, accuracy ranged from 35.4% to 85.9%, with approximately half passing. Among the evaluated model versions, adjusted accuracy differed substantially when models were grouped by family (for example, Claude-family models, 84% vs. GPT-4, 43%); these family-grouped comparisons were descriptive because only 16 individual models were evaluated. Paired prompt analyses showed modest average absolute effects, ranging from a 2.03-percentage-point decrease with implicit CoT to a 3.04-percentage-point increase with explicit CoT. However, prompt–model interactions produced substantial model-level divergence: prompt optimization reversed failing performance in one model (87.4% vs. 49.7%) while reducing accuracy in others. No single prompt strategy yielded consistent benefits across the evaluated models. Among five selected lower-performing models evaluated under explicit CoT prompting, misunderstanding of foundational TCM concepts was the most frequent error. These findings indicate that reliable LLM deployment in TCM requires empirical validation of both model choice and prompt strategy: model selection strongly shapes baseline capability, while prompt-model interactions can materially alter performance.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Tseng, L.-W., Lu, Y.-C., Tseng, L.-C., Chen, Y., & Chen, H.-Y. (2026). Model-dependent prompt effects challenge reliable deployment of medical LLMs in traditional Chinese medicine. npj Digital Medicine. https://doi.org/10.1038/s41746-026-03397-5
MLA 9
Tseng, Liang-Wei, et al. “Model-dependent prompt effects challenge reliable deployment of medical LLMs in traditional Chinese medicine.” npj Digital Medicine, 2026. https://doi.org/10.1038/s41746-026-03397-5.
Chicago
Tseng, Liang-Wei, Yi-Chin Lu, Liang-Chi Tseng, Yu‐Chun Chen, and Hsing-Yu Chen. 2026. “Model-dependent prompt effects challenge reliable deployment of medical LLMs in traditional Chinese medicine.” npj Digital Medicine. https://doi.org/10.1038/s41746-026-03397-5.
Harvard
Tseng, L.-W. et al. (2026) ‘Model-dependent prompt effects challenge reliable deployment of medical LLMs in traditional Chinese medicine’, npj Digital Medicine. doi:10.1038/s41746-026-03397-5.
IEEE
L.-W. Tseng, Y.-C. Lu, L.-C. Tseng, Y. Chen, and H.-Y. Chen, “Model-dependent prompt effects challenge reliable deployment of medical LLMs in traditional Chinese medicine,” npj Digital Medicine, 2026, doi: 10.1038/s41746-026-03397-5.
BibTeX
@article{tseng2026model,
title = {{Model-dependent prompt effects challenge reliable deployment of medical LLMs in traditional Chinese medicine}},
author = {Tseng, Liang-Wei and Lu, Yi-Chin and Tseng, Liang-Chi and Chen, Yu‐Chun and Chen, Hsing-Yu},
journal = {npj Digital Medicine},
year = {2026},
doi = {10.1038/s41746-026-03397-5},
url = {https://doi.org/10.1038/s41746-026-03397-5}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- large language models
- traditional Chinese medicine
- prompt engineering
- chain-of-thought prompting
- model selection
- medical knowledge assessment
Work with this paper
Most-cited papers in Medicine
- Yaşlı Kadınlarda Üreme SağlığıDemet Kokanalı, Yaprak Engin Üstün · Jinekoloji-Obstetrik ve Neonatoloji Tıp Dergisi · 2019 · 169,630 citations
- Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 CountriesHyuna Sung, Adalberto M. Filho, Mathieu Laversanne and 5 more · CA A Cancer Journal for Clinicians · 2021 · 119,885 citations
- “Mini-mental state”Marshal F. Folstein, Susan E. Folstein, Paul R. McHugh · Journal of Psychiatric Research · 1975 · 90,424 citations
- Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countriesFreddie Bray, Jacques Ferlay, Isabelle Soerjomataram and 3 more · CA A Cancer Journal for Clinicians · 2018 · 87,396 citations
- Hallmarks of Cancer: The Next GenerationDouglas Hanahan, Robert A. Weinberg · Cell · 2011 · 69,758 citations
Questions about this paper
How do I cite this paper in APA style?
Tseng, L.-W., Lu, Y.-C., Tseng, L.-C., Chen, Y., & Chen, H.-Y. (2026). Model-dependent prompt effects challenge reliable deployment of medical LLMs in traditional Chinese medicine. npj Digital Medicine. https://doi.org/10.1038/s41746-026-03397-5
How many times has this paper been cited?
1 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at npj Digital Medicine: sign in and open the PDF from this page.
Who wrote this paper?
Liang-Wei Tseng, Yi-Chin Lu, Liang-Chi Tseng, Yu‐Chun Chen, Hsing-Yu Chen.
What is the DOI of this paper?
10.1038/s41746-026-03397-5, permanently at https://doi.org/10.1038/s41746-026-03397-5. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.