Conference paper · 2019 · Open access

Private Model Compression via Knowledge Distillation

Ji Wang, Weidong Bao, Lichao Sun, Xiaomin Zhu, Bokai Cao, Philip S. Yu

Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 1190–1197 · Published 17 July 2019 · DOI 10.1609/aaai.v33i01.33011190

Open PDFView at the publisherCite this paper

In brief

“Private Model Compression via Knowledge Distillation” is a 2019 conference paper by Ji Wang and 5 co-authors, published in Proceedings of the AAAI Conference on Artificial Intelligence. OpenAlex counts 150 citations as of 19 November 2026, which puts it in the top 2% of works published in 2019 by citations. It is open access: the full text is free to read at Proceedings of the AAAI Conference on Artificial Intelligence.

Citations
150
Most cited in
2021 (35)
Field-weighted impact
5.30
References
37
Authors
6

Abstract

The soaring demand for intelligent mobile applications calls for deploying powerful deep neural networks (DNNs) on mobile devices. However, the outstanding performance of DNNs notoriously relies on increasingly complex models, which in turn is associated with an increase in computational expense far surpassing mobile devices’ capacity. What is worse, app service providers need to collect and utilize a large volume of users’ data, which contain sensitive information, to build the sophisticated DNN models. Directly deploying these models on public mobile devices presents prohibitive privacy risk. To benefit from the on-device deep learning without the capacity and privacy concerns, we design a private model compression framework RONA. Following the knowledge distillation paradigm, we jointly use hint learning, distillation learning, and self learning to train a compact and fast neural network. The knowledge distilled from the cumbersome model is adaptively bounded and carefully perturbed to enforce differential privacy. We further propose an elegant query sample selection method to reduce the number of queries and control the privacy loss. A series of empirical evaluations as well as the implementation on an Android mobile device show that RONA can not only compress cumbersome models efficiently but also provide a strong privacy guarantee. For example, on SVHN, when a meaningful (9.83,10−6)-differential privacy is guaranteed, the compact model trained by RONA can obtain 20× compression ratio and 19× speed-up with merely 0.97% accuracy loss.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Wang, J., Bao, W., Sun, L., Zhu, X., Cao, B., & Yu, P. S. (2019). Private Model Compression via Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 1190–1197. https://doi.org/10.1609/aaai.v33i01.33011190

MLA 9

Wang, Ji, et al. “Private Model Compression via Knowledge Distillation.” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 1190–1197. https://doi.org/10.1609/aaai.v33i01.33011190.

Chicago

Wang, Ji, Weidong Bao, Lichao Sun, Xiaomin Zhu, Bokai Cao, and Philip S. Yu. 2019. “Private Model Compression via Knowledge Distillation.” Proceedings of the AAAI Conference on Artificial Intelligence 33 (01): 1190–1197. https://doi.org/10.1609/aaai.v33i01.33011190.

Harvard

Wang, J. et al. (2019) ‘Private Model Compression via Knowledge Distillation’, Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), pp. 1190–1197. doi:10.1609/aaai.v33i01.33011190.

IEEE

J. Wang, W. Bao, L. Sun, X. Zhu, B. Cao, and P. S. Yu, “Private Model Compression via Knowledge Distillation,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 1190–1197, 2019, doi: 10.1609/aaai.v33i01.33011190.

BibTeX

@inproceedings{wang2019private,
  title     = {{Private Model Compression via Knowledge Distillation}},
  author    = {Wang, Ji and Bao, Weidong and Sun, Lichao and Zhu, Xiaomin and Cao, Bokai and Yu, Philip S.},
  booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year      = {2019},
  volume    = {33},
  number    = {01},
  pages     = {1190--1197},
  doi       = {10.1609/aaai.v33i01.33011190},
  url       = {https://doi.org/10.1609/aaai.v33i01.33011190}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
201966
20201717
20213535
20221515
20232020
20242626
20252525
202666

Its field-weighted citation impact is 5.30: it has been cited 5.30 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • knowledge distillation
  • differential privacy
  • on-device deep learning
  • mobile devices
  • self-supervised learning
  • privacy-preserving neural networks
  • Rona

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Wang, J., Bao, W., Sun, L., Zhu, X., Cao, B., & Yu, P. S. (2019). Private Model Compression via Knowledge Distillation. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 1190–1197. https://doi.org/10.1609/aaai.v33i01.33011190

How many times has this paper been cited?

150 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Proceedings of the AAAI Conference on Artificial Intelligence: sign in and open the PDF from this page.

Who wrote this paper?

Ji Wang, Weidong Bao, Lichao Sun, Xiaomin Zhu, Bokai Cao, Philip S. Yu.

What is the DOI of this paper?

10.1609/aaai.v33i01.33011190, permanently at https://doi.org/10.1609/aaai.v33i01.33011190. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.