Conference paper · 2022 · Open access
Differentially Private Model Compression
neural information processing systems, 29468–29483 · Published 1 January 2022 · DOI 10.52202/068431-2137
In brief
“Differentially Private Model Compression” is a 2022 conference paper by Fatemehsadat Mireshghallah and 4 co-authors, published in neural information processing systems. OpenAlex counts 26 citations as of 19 November 2026, which puts it in the top 3% of works published in 2022 by citations. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 26
- Most cited in
- 2023 (8)
- Field-weighted impact
- 1.49
- References
- 52
- Authors
- 5
Abstract
Recent papers have shown that large pre-trained language models (LLMs) such as BERT, GPT-2 can be fine-tuned on private data to achieve performance comparable to non-private models for many downstream Natural Language Processing (NLP) tasks while simultaneously guaranteeing differential privacy. The inference cost of these models -- which consist of hundreds of millions of parameters -- however, can be prohibitively large. Hence, often in practice, LLMs are compressed before they are deployed in specific applications. In this paper, we initiate the study of differentially private model compression and propose frameworks for achieving 50% sparsity levels while maintaining nearly full performance. We demonstrate these ideas on standard GLUE benchmarks using BERT models, setting benchmarks for future research on this topic.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Mireshghallah, F., Bačkurs, A., Inan, H. A., Wutschitz, L., & Kulkarni, J. (2022). Differentially Private Model Compression. neural information processing systems, 29468–29483. https://doi.org/10.52202/068431-2137
MLA 9
Mireshghallah, Fatemehsadat, et al. “Differentially Private Model Compression.” neural information processing systems, 2022, pp. 29468–29483. https://doi.org/10.52202/068431-2137.
Chicago
Mireshghallah, Fatemehsadat, Artūrs Bačkurs, Huseyin A. Inan, Lukas Wutschitz, and Janardhan Kulkarni. 2022. “Differentially Private Model Compression.” neural information processing systems: 29468–29483. https://doi.org/10.52202/068431-2137.
Harvard
Mireshghallah, F. et al. (2022) ‘Differentially Private Model Compression’, neural information processing systems, pp. 29468–29483. doi:10.52202/068431-2137.
IEEE
F. Mireshghallah, A. Bačkurs, H. A. Inan, L. Wutschitz, and J. Kulkarni, “Differentially Private Model Compression,” neural information processing systems, pp. 29468–29483, 2022, doi: 10.52202/068431-2137.
BibTeX
@inproceedings{mireshghallah2022differentially,
title = {{Differentially Private Model Compression}},
author = {Mireshghallah, Fatemehsadat and Bačkurs, Artūrs and Inan, Huseyin A. and Wutschitz, Lukas and Kulkarni, Janardhan},
booktitle = {neural information processing systems},
year = {2022},
pages = {29468--29483},
doi = {10.52202/068431-2137},
url = {https://doi.org/10.52202/068431-2137}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2023 | 8 | |
| 2024 | 8 | |
| 2025 | 6 | |
| 2026 | 4 |
Its field-weighted citation impact is 1.49: it has been cited 1.49 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- differential privacy
- model compression
- sparsity
- pre-trained language models
- BERT
- inference cost
- GLUE benchmark
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Mireshghallah, F., Bačkurs, A., Inan, H. A., Wutschitz, L., & Kulkarni, J. (2022). Differentially Private Model Compression. neural information processing systems, 29468–29483. https://doi.org/10.52202/068431-2137
How many times has this paper been cited?
26 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Fatemehsadat Mireshghallah, Artūrs Bačkurs, Huseyin A. Inan, Lukas Wutschitz, Janardhan Kulkarni.
What is the DOI of this paper?
10.52202/068431-2137, permanently at https://doi.org/10.52202/068431-2137. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.