Conference paper · 2022 · Open access

Differentially Private Model Compression

Fatemehsadat Mireshghallah, Artūrs Bačkurs, Huseyin A. Inan, Lukas Wutschitz, Janardhan Kulkarni

neural information processing systems, 29468–29483 · Published 1 January 2022 · DOI 10.52202/068431-2137

Open PDFView at the publisherCite this paper

In brief

“Differentially Private Model Compression” is a 2022 conference paper by Fatemehsadat Mireshghallah and 4 co-authors, published in neural information processing systems. OpenAlex counts 26 citations as of 19 November 2026, which puts it in the top 3% of works published in 2022 by citations. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).

Citations
26
Most cited in
2023 (8)
Field-weighted impact
1.49
References
52
Authors
5

Abstract

Recent papers have shown that large pre-trained language models (LLMs) such as BERT, GPT-2 can be fine-tuned on private data to achieve performance comparable to non-private models for many downstream Natural Language Processing (NLP) tasks while simultaneously guaranteeing differential privacy. The inference cost of these models -- which consist of hundreds of millions of parameters -- however, can be prohibitively large. Hence, often in practice, LLMs are compressed before they are deployed in specific applications. In this paper, we initiate the study of differentially private model compression and propose frameworks for achieving 50% sparsity levels while maintaining nearly full performance. We demonstrate these ideas on standard GLUE benchmarks using BERT models, setting benchmarks for future research on this topic.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Mireshghallah, F., Bačkurs, A., Inan, H. A., Wutschitz, L., & Kulkarni, J. (2022). Differentially Private Model Compression. neural information processing systems, 29468–29483. https://doi.org/10.52202/068431-2137

MLA 9

Mireshghallah, Fatemehsadat, et al. “Differentially Private Model Compression.” neural information processing systems, 2022, pp. 29468–29483. https://doi.org/10.52202/068431-2137.

Chicago

Mireshghallah, Fatemehsadat, Artūrs Bačkurs, Huseyin A. Inan, Lukas Wutschitz, and Janardhan Kulkarni. 2022. “Differentially Private Model Compression.” neural information processing systems: 29468–29483. https://doi.org/10.52202/068431-2137.

Harvard

Mireshghallah, F. et al. (2022) ‘Differentially Private Model Compression’, neural information processing systems, pp. 29468–29483. doi:10.52202/068431-2137.

IEEE

F. Mireshghallah, A. Bačkurs, H. A. Inan, L. Wutschitz, and J. Kulkarni, “Differentially Private Model Compression,” neural information processing systems, pp. 29468–29483, 2022, doi: 10.52202/068431-2137.

BibTeX

@inproceedings{mireshghallah2022differentially,
  title     = {{Differentially Private Model Compression}},
  author    = {Mireshghallah, Fatemehsadat and Bačkurs, Artūrs and Inan, Huseyin A. and Wutschitz, Lukas and Kulkarni, Janardhan},
  booktitle = {neural information processing systems},
  year      = {2022},
  pages     = {29468--29483},
  doi       = {10.52202/068431-2137},
  url       = {https://doi.org/10.52202/068431-2137}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202388
202488
202566
202644

Its field-weighted citation impact is 1.49: it has been cited 1.49 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • differential privacy
  • model compression
  • sparsity
  • pre-trained language models
  • BERT
  • inference cost
  • GLUE benchmark

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Mireshghallah, F., Bačkurs, A., Inan, H. A., Wutschitz, L., & Kulkarni, J. (2022). Differentially Private Model Compression. neural information processing systems, 29468–29483. https://doi.org/10.52202/068431-2137

How many times has this paper been cited?

26 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Fatemehsadat Mireshghallah, Artūrs Bačkurs, Huseyin A. Inan, Lukas Wutschitz, Janardhan Kulkarni.

What is the DOI of this paper?

10.52202/068431-2137, permanently at https://doi.org/10.52202/068431-2137. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.