Journal article · 2022 · Open access
Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation
IEEE Transactions on Image Processing, 32, 116–127 · Published 7 December 2022 · DOI 10.1109/tip.2022.3226416
In brief
“Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation” is a 2022 journal article by Shiming Ge and 4 co-authors, published in IEEE Transactions on Image Processing. OpenAlex counts 28 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 28
- Most cited in
- 2025 (15)
- Field-weighted impact
- 1.29
- References
- 82
- Authors
- 5
Abstract
While deep models have proved successful in learning rich knowledge from massive well-annotated data, they may pose a privacy leakage risk in practical deployment. It is necessary to find an effective trade-off between high utility and strong privacy. In this work, we propose a discriminative-generative distillation approach to learn privacy-preserving deep models. Our key idea is taking models as bridge to distill knowledge from private data and then transfer it to learn a student network via two streams. First, discriminative stream trains a baseline classifier on private data and an ensemble of teachers on multiple disjoint private subsets, respectively. Then, generative stream takes the classifier as a fixed discriminator and trains a generator in a data-free manner. After that, the generator is used to generate massive synthetic data which are further applied to train a variational autoencoder (VAE). Among these synthetic data, a few of them are fed into the teacher ensemble to query labels via differentially private aggregation, while most of them are embedded to the trained VAE for reconstructing synthetic data. Finally, a semi-supervised student learning is performed to simultaneously handle two tasks: knowledge transfer from the teachers with distillation on few privately labeled synthetic data, and knowledge enhancement with tangent-normal adversarial regularization on many triples of reconstructed synthetic data. In this way, our approach can control query cost over private data and mitigate accuracy degradation in a unified manner, leading to a privacy-preserving student model. Extensive experiments and analysis clearly show the effectiveness of the proposed approach.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Ge, S., Liu, B., Wang, P., Li, Y., & Zeng, D. (2022). Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation. IEEE Transactions on Image Processing, 32, 116–127. https://doi.org/10.1109/tip.2022.3226416
MLA 9
Ge, Shiming, et al. “Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation.” IEEE Transactions on Image Processing, vol. 32, 2022, pp. 116–127. https://doi.org/10.1109/tip.2022.3226416.
Chicago
Ge, Shiming, Bochao Liu, Pengju Wang, Yong Li, and Dan Zeng. 2022. “Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation.” IEEE Transactions on Image Processing 32: 116–127. https://doi.org/10.1109/tip.2022.3226416.
Harvard
Ge, S. et al. (2022) ‘Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation’, IEEE Transactions on Image Processing, 32, pp. 116–127. doi:10.1109/tip.2022.3226416.
IEEE
S. Ge, B. Liu, P. Wang, Y. Li, and D. Zeng, “Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation,” IEEE Transactions on Image Processing, vol. 32, pp. 116–127, 2022, doi: 10.1109/tip.2022.3226416.
BibTeX
@article{ge2022learning,
title = {{Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation}},
author = {Ge, Shiming and Liu, Bochao and Wang, Pengju and Li, Yong and Zeng, Dan},
journal = {IEEE Transactions on Image Processing},
year = {2022},
volume = {32},
pages = {116--127},
doi = {10.1109/tip.2022.3226416},
url = {https://doi.org/10.1109/tip.2022.3226416}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2023 | 1 | |
| 2024 | 7 | |
| 2025 | 15 | |
| 2026 | 5 |
Its field-weighted citation impact is 1.29: it has been cited 1.29 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- privacy-preserving deep learning
- knowledge distillation
- synthetic data generation
- variational autoencoder
- differential privacy
- semi-supervised learning
- adversarial regularization
- privacy leakage
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Ge, S., Liu, B., Wang, P., Li, Y., & Zeng, D. (2022). Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation. IEEE Transactions on Image Processing, 32, 116–127. https://doi.org/10.1109/tip.2022.3226416
How many times has this paper been cited?
28 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Shiming Ge, Bochao Liu, Pengju Wang, Yong Li, Dan Zeng.
What is the DOI of this paper?
10.1109/tip.2022.3226416, permanently at https://doi.org/10.1109/tip.2022.3226416. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.