Preprint · 2026 · Open access
DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models
arXiv (Cornell University) · Published 17 May 2026 · DOI 10.48550/arxiv.2605.17432
In brief
“DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models” is a 2026 preprint by Haichao Sha and 4 co-authors, published in arXiv (Cornell University). OpenAlex counts 0 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 0
- References
- 40
- Authors
- 5
Abstract
Large language models (LLMs) are commonly adapted to downstream tasks through fine-tuning, but fine-tuning data often contains sensitive information that may be leaked by the resulting model. Differential privacy (DP) offers formal protection against such leakage, yet DP fine-tuning of LLMs still suffers from substantial utility degradation due to gradient clipping and noise injection. Existing work improves this trade-off by combining DP with parameter-efficient fine-tuning methods such as LoRA, which constrain the form of updates. In this work, we study a complementary direction: selective fine-tuning, which constrains where updates are applied. We propose DP-SelFT, a framework for differentially private selective fine-tuning of LLMs. DP-SelFT addresses three DP-specific challenges in parameter selection: avoiding repeated privacy cost, improving stability under noisy estimates, and selecting parameters that remain useful under clipped and noisy updates. It first constructs a lightweight DP synthetic dataset and performs selection only on this synthetic data, so the selection stage incurs no additional privacy cost. It then conducts layer-level selection by temporarily training candidate layer subsets on a synthetic training split and evaluating them on a synthetic validation split. Crucially, this temporary training is performed under a perturbation regime matched to downstream DP fine-tuning, with worst-case perturbations of the same scale as DP noise. This favors layer subsets that are not only learnable but also robust to noisy private updates. Experiments on benchmark tasks show that DP-SelFT consistently improves the privacy--utility trade-off over existing DP fine-tuning baselines under the same privacy guarantees.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Sha, H., Wang, Z., Wu, Y., Chen, H., & Dong, W. (2026). DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.17432
MLA 9
Sha, Haichao, et al. “DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2605.17432.
Chicago
Sha, Haichao, Zihao Wang, Yuncheng Wu, hong Chen, and Wei Dong. 2026. “DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.17432.
Harvard
Sha, H. et al. (2026) ‘DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models’, arXiv (Cornell University). doi:10.48550/arxiv.2605.17432.
IEEE
H. Sha, Z. Wang, Y. Wu, H. Chen, and W. Dong, “DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2605.17432.
BibTeX
@misc{sha2026selft,
title = {{DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models}},
author = {Sha, Haichao and Wang, Zihao and Wu, Yuncheng and Chen, hong and Dong, Wei},
howpublished = {arXiv (Cornell University)},
year = {2026},
doi = {10.48550/arxiv.2605.17432},
url = {https://doi.org/10.48550/arxiv.2605.17432}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- differential privacy
- selective fine-tuning
- large language models
- privacy-utility trade-off
- synthetic training data
- layer selection
- parameter selection
- gradient clipping
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Sha, H., Wang, Z., Wu, Y., Chen, H., & Dong, W. (2026). DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.17432
How many times has this paper been cited?
0 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Haichao Sha, Zihao Wang, Yuncheng Wu, hong Chen, Wei Dong.
What is the DOI of this paper?
10.48550/arxiv.2605.17432, permanently at https://doi.org/10.48550/arxiv.2605.17432. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.