Preprint · 2025 · Open access

How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding

Dufour, Mathieu, A. Duncan

arXiv (Cornell University) · Published 18 November 2025 · DOI 10.48550/arxiv.2511.14936

Open PDFView at the publisherCite this paper

In brief

“How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding” is a 2025 preprint by Dufour, Mathieu and A. Duncan, published in arXiv (Cornell University). OpenAlex counts 0 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).

Citations
0
References
17
Authors
2

Abstract

Large language models trained on clinical text risk exposing sensitive patient information, yet differential privacy (DP) methods often severely degrade the diagnostic accuracy needed for deployment. Despite rapid progress in DP optimisation and text generation, it remains unclear which privacy-preserving strategy actually works best for clinical language tasks. We present the first systematic head-to-head comparison of four training pipelines for automated diagnostic coding from hospital discharge summaries. All pipelines use identical 1B-parameter models and matched privacy budgets to predict ICD-9 codes. At moderate and relaxed privacy budgets ($\varepsilon \in \{4, 6\}$), knowledge distillation from DP-trained teachers outperforms both direct DP-SGD and DP-synthetic data training, recovering up to 63\% of the non-private performance whilst maintaining strong empirical privacy (membership-inference AUC $\approx$ 0.5). These findings expose large differences in the privacy-utility trade-off across architectures and identify knowledge distillation as the most practical route to privacy-preserving clinical NLP.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Mathieu, D., & Duncan, A. (2025). How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2511.14936

MLA 9

Mathieu, Dufour,, and A. Duncan. “How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2511.14936.

Chicago

Mathieu, Dufour, and A. Duncan. 2025. “How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2511.14936.

Harvard

Mathieu, D. and Duncan, A. (2025) ‘How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding’, arXiv (Cornell University). doi:10.48550/arxiv.2511.14936.

IEEE

D. Mathieu and A. Duncan, “How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2511.14936.

BibTeX

@misc{mathieu2025train,
  title     = {{How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding}},
  author    = {Mathieu, Dufour, and Duncan, A.},
  howpublished = {arXiv (Cornell University)},
  year      = {2025},
  doi       = {10.48550/arxiv.2511.14936},
  url       = {https://doi.org/10.48550/arxiv.2511.14936}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • differential privacy
  • clinical language models
  • ICD-9 coding
  • discharge summaries
  • knowledge distillation
  • DP-SGD
  • synthetic data training
  • privacy-utility trade-off

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Mathieu, D., & Duncan, A. (2025). How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2511.14936

How many times has this paper been cited?

0 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Dufour, Mathieu, A. Duncan.

What is the DOI of this paper?

10.48550/arxiv.2511.14936, permanently at https://doi.org/10.48550/arxiv.2511.14936. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.