Journal article · 2022 · Open access

Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation

Shiming Ge, Bochao Liu, Pengju Wang, Yong Li, Dan Zeng

IEEE Transactions on Image Processing, 32, 116–127 · Published 7 December 2022 · DOI 10.1109/tip.2022.3226416

Open PDFView at the publisherCite this paper

In brief

“Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation” is a 2022 journal article by Shiming Ge and 4 co-authors, published in IEEE Transactions on Image Processing. OpenAlex counts 28 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
28
Most cited in
2025 (15)
Field-weighted impact
1.29
References
82
Authors
5

Abstract

While deep models have proved successful in learning rich knowledge from massive well-annotated data, they may pose a privacy leakage risk in practical deployment. It is necessary to find an effective trade-off between high utility and strong privacy. In this work, we propose a discriminative-generative distillation approach to learn privacy-preserving deep models. Our key idea is taking models as bridge to distill knowledge from private data and then transfer it to learn a student network via two streams. First, discriminative stream trains a baseline classifier on private data and an ensemble of teachers on multiple disjoint private subsets, respectively. Then, generative stream takes the classifier as a fixed discriminator and trains a generator in a data-free manner. After that, the generator is used to generate massive synthetic data which are further applied to train a variational autoencoder (VAE). Among these synthetic data, a few of them are fed into the teacher ensemble to query labels via differentially private aggregation, while most of them are embedded to the trained VAE for reconstructing synthetic data. Finally, a semi-supervised student learning is performed to simultaneously handle two tasks: knowledge transfer from the teachers with distillation on few privately labeled synthetic data, and knowledge enhancement with tangent-normal adversarial regularization on many triples of reconstructed synthetic data. In this way, our approach can control query cost over private data and mitigate accuracy degradation in a unified manner, leading to a privacy-preserving student model. Extensive experiments and analysis clearly show the effectiveness of the proposed approach.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Ge, S., Liu, B., Wang, P., Li, Y., & Zeng, D. (2022). Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation. IEEE Transactions on Image Processing, 32, 116–127. https://doi.org/10.1109/tip.2022.3226416

MLA 9

Ge, Shiming, et al. “Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation.” IEEE Transactions on Image Processing, vol. 32, 2022, pp. 116–127. https://doi.org/10.1109/tip.2022.3226416.

Chicago

Ge, Shiming, Bochao Liu, Pengju Wang, Yong Li, and Dan Zeng. 2022. “Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation.” IEEE Transactions on Image Processing 32: 116–127. https://doi.org/10.1109/tip.2022.3226416.

Harvard

Ge, S. et al. (2022) ‘Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation’, IEEE Transactions on Image Processing, 32, pp. 116–127. doi:10.1109/tip.2022.3226416.

IEEE

S. Ge, B. Liu, P. Wang, Y. Li, and D. Zeng, “Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation,” IEEE Transactions on Image Processing, vol. 32, pp. 116–127, 2022, doi: 10.1109/tip.2022.3226416.

BibTeX

@article{ge2022learning,
  title     = {{Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation}},
  author    = {Ge, Shiming and Liu, Bochao and Wang, Pengju and Li, Yong and Zeng, Dan},
  journal   = {IEEE Transactions on Image Processing},
  year      = {2022},
  volume    = {32},
  pages     = {116--127},
  doi       = {10.1109/tip.2022.3226416},
  url       = {https://doi.org/10.1109/tip.2022.3226416}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202311
202477
20251515
202655

Its field-weighted citation impact is 1.29: it has been cited 1.29 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • privacy-preserving deep learning
  • knowledge distillation
  • synthetic data generation
  • variational autoencoder
  • differential privacy
  • semi-supervised learning
  • adversarial regularization
  • privacy leakage

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Ge, S., Liu, B., Wang, P., Li, Y., & Zeng, D. (2022). Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation. IEEE Transactions on Image Processing, 32, 116–127. https://doi.org/10.1109/tip.2022.3226416

How many times has this paper been cited?

28 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Shiming Ge, Bochao Liu, Pengju Wang, Yong Li, Dan Zeng.

What is the DOI of this paper?

10.1109/tip.2022.3226416, permanently at https://doi.org/10.1109/tip.2022.3226416. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.