Journal article · 2022 · Open access
PKDGAN: Private Knowledge Distillation With Generative Adversarial Networks
IEEE Transactions on Big Data, 10(6), 775–788 · Published 4 November 2022 · DOI 10.1109/tbdata.2022.3216566
In brief
“PKDGAN: Private Knowledge Distillation With Generative Adversarial Networks” is a 2022 journal article by Cheng Zhuo and 2 co-authors, published in IEEE Transactions on Big Data. OpenAlex counts 12 citations as of 19 November 2026. It is open access under a public domain licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 12
- Most cited in
- 2025 (6)
- Field-weighted impact
- 0.53
- References
- 56
- Authors
- 3
Abstract
The deployment of deep learning applications has to address the increasing privacy concerns when using private and sensitive data for training. A conventional deep learning model is prone to privacy attacks that can recover the sensitive information from either model parameters or accesses to the inference model. Recently, differential privacy (DP) has been proposed to offer provable privacy guarantees by randomizing the training process of neural networks. However, many approaches tend to provide the worst case privacy protection for model publishing, inevitably impairing the accuracy of the trained models. Thus, we present a novel private knowledge transfer strategy, where the private teacher trained on sensitive data is not publicly accessible but the student models can be released with privacy guarantees. In this paper, a three-player (teacher-student-discriminator) learning framework, Private Knowledge Distillation with Generative Adversarial Networks (PKDGAN), is proposed, where the student acquires the distilled knowledge from the teacher and is trained with the discriminator to generate similar outputs as the teacher. Moreover, a cooperative learning strategy is also suggested to support the collective training of multiple students against the discriminator when each student is with insufficient unlabelled training data. To enforce rigorous privacy guarantees, PKDGAN applies a Rényi differential privacy mechanism throughout the training process, and use it with a moment accountant technique to track the privacy cost. PKDGAN allows students to be trained with unlabelled public data and very few epochs, which avoids the exposure of training data while ensuring model performance. In the experiments, PKDGAN is found to have consistently good performance on various datasets (MNIST, SVHN, CIFAR-10, and Market-1501). When compared to prior works [1], [2], PKDGAN exhibits 5-82% accuracy loss improvement without compromising any privacy guarantee.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Zhuo, C., Gao, D., & Liu, L. (2022). PKDGAN: Private Knowledge Distillation With Generative Adversarial Networks. IEEE Transactions on Big Data, 10(6), 775–788. https://doi.org/10.1109/tbdata.2022.3216566
MLA 9
Zhuo, Cheng, et al. “PKDGAN: Private Knowledge Distillation With Generative Adversarial Networks.” IEEE Transactions on Big Data, vol. 10, no. 6, 2022, pp. 775–788. https://doi.org/10.1109/tbdata.2022.3216566.
Chicago
Zhuo, Cheng, Di Gao, and Liangwei Liu. 2022. “PKDGAN: Private Knowledge Distillation With Generative Adversarial Networks.” IEEE Transactions on Big Data 10 (6): 775–788. https://doi.org/10.1109/tbdata.2022.3216566.
Harvard
Zhuo, C., Gao, D. and Liu, L. (2022) ‘PKDGAN: Private Knowledge Distillation With Generative Adversarial Networks’, IEEE Transactions on Big Data, 10(6), pp. 775–788. doi:10.1109/tbdata.2022.3216566.
IEEE
C. Zhuo, D. Gao, and L. Liu, “PKDGAN: Private Knowledge Distillation With Generative Adversarial Networks,” IEEE Transactions on Big Data, vol. 10, no. 6, pp. 775–788, 2022, doi: 10.1109/tbdata.2022.3216566.
BibTeX
@article{zhuo2022pkdgan,
title = {{PKDGAN: Private Knowledge Distillation With Generative Adversarial Networks}},
author = {Zhuo, Cheng and Gao, Di and Liu, Liangwei},
journal = {IEEE Transactions on Big Data},
year = {2022},
volume = {10},
number = {6},
pages = {775--788},
doi = {10.1109/tbdata.2022.3216566},
url = {https://doi.org/10.1109/tbdata.2022.3216566}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2022 | 1 | |
| 2023 | 1 | |
| 2024 | 1 | |
| 2025 | 6 | |
| 2026 | 3 |
Its field-weighted citation impact is 0.53: it has been cited 0.53 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- generative adversarial networks
- differential privacy
- Rényi differential privacy
- privacy-preserving machine learning
- cooperative learning
- unlabeled data
- sensitive data protection
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Zhuo, C., Gao, D., & Liu, L. (2022). PKDGAN: Private Knowledge Distillation With Generative Adversarial Networks. IEEE Transactions on Big Data, 10(6), 775–788. https://doi.org/10.1109/tbdata.2022.3216566
How many times has this paper been cited?
12 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Cheng Zhuo, Di Gao, Liangwei Liu.
What is the DOI of this paper?
10.1109/tbdata.2022.3216566, permanently at https://doi.org/10.1109/tbdata.2022.3216566. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.