Preprint · 2025 · Open access
How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding
arXiv (Cornell University) · Published 18 November 2025 · DOI 10.48550/arxiv.2511.14936
In brief
“How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding” is a 2025 preprint by Dufour, Mathieu and A. Duncan, published in arXiv (Cornell University). OpenAlex counts 0 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 0
- References
- 17
- Authors
- 2
Abstract
Large language models trained on clinical text risk exposing sensitive patient information, yet differential privacy (DP) methods often severely degrade the diagnostic accuracy needed for deployment. Despite rapid progress in DP optimisation and text generation, it remains unclear which privacy-preserving strategy actually works best for clinical language tasks. We present the first systematic head-to-head comparison of four training pipelines for automated diagnostic coding from hospital discharge summaries. All pipelines use identical 1B-parameter models and matched privacy budgets to predict ICD-9 codes. At moderate and relaxed privacy budgets ($\varepsilon \in \{4, 6\}$), knowledge distillation from DP-trained teachers outperforms both direct DP-SGD and DP-synthetic data training, recovering up to 63\% of the non-private performance whilst maintaining strong empirical privacy (membership-inference AUC $\approx$ 0.5). These findings expose large differences in the privacy-utility trade-off across architectures and identify knowledge distillation as the most practical route to privacy-preserving clinical NLP.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Mathieu, D., & Duncan, A. (2025). How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2511.14936
MLA 9
Mathieu, Dufour,, and A. Duncan. “How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2511.14936.
Chicago
Mathieu, Dufour, and A. Duncan. 2025. “How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2511.14936.
Harvard
Mathieu, D. and Duncan, A. (2025) ‘How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding’, arXiv (Cornell University). doi:10.48550/arxiv.2511.14936.
IEEE
D. Mathieu and A. Duncan, “How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2511.14936.
BibTeX
@misc{mathieu2025train,
title = {{How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding}},
author = {Mathieu, Dufour, and Duncan, A.},
howpublished = {arXiv (Cornell University)},
year = {2025},
doi = {10.48550/arxiv.2511.14936},
url = {https://doi.org/10.48550/arxiv.2511.14936}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- differential privacy
- clinical language models
- ICD-9 coding
- discharge summaries
- knowledge distillation
- DP-SGD
- synthetic data training
- privacy-utility trade-off
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Mathieu, D., & Duncan, A. (2025). How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2511.14936
How many times has this paper been cited?
0 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Dufour, Mathieu, A. Duncan.
What is the DOI of this paper?
10.48550/arxiv.2511.14936, permanently at https://doi.org/10.48550/arxiv.2511.14936. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.