Conference paper · 2022 · Open access

Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation

Bochao Liu, Jianghu Lu, Pengju Wang, Junjie Zhang, Dan Zeng, Zhenxing Qian, Shiming Ge

IEEE International Workshop on Multimedia Signal Processing (MMSP), 01–06 · Published 26 September 2022 · DOI 10.1109/mmsp55362.2022.9950001

Open PDFView at the publisherCite this paper

In brief

“Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation” is a 2022 conference paper by Bochao Liu and 6 co-authors, published in IEEE International Workshop on Multimedia Signal Processing (MMSP). OpenAlex counts 5 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
5
Most cited in
2025 (2)
Field-weighted impact
0.27
References
33
Authors
7

Abstract

Deep learning models can achieve high inference accuracy by extracting rich knowledge from massive well-annotated data, but may pose the risk of data privacy leakage in practical deployment. In this paper, we present an effective teacher-student learning approach to train privacy-preserving deep learning models via differentially private data-free distillation. The main idea is generating synthetic data to learn a student that can mimic the ability of a teacher well-trained on private data. In the approach, a generator is first pretrained in a data-free manner by incorporating the teacher as a fixed discriminator. With the generator, massive synthetic data can be generated for model training without exposing data privacy. Then, the synthetic data is fed into the teacher to generate private labels. Towards this end, we propose a label differential privacy algorithm termed selective randomized response to protect the label information. Finally, a student is trained on the synthetic data with the supervision of private labels. In this way, both data privacy and label privacy are well protected in a unified framework, leading to privacy-preserving models. Extensive experiments and analysis clearly demonstrate the effectiveness of our approach.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Liu, B., Lu, J., Wang, P., Zhang, J., Zeng, D., Qian, Z., & Ge, S. (2022). Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation. IEEE International Workshop on Multimedia Signal Processing (MMSP), 01–06. https://doi.org/10.1109/mmsp55362.2022.9950001

MLA 9

Liu, Bochao, et al. “Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation.” IEEE International Workshop on Multimedia Signal Processing (MMSP), 2022, pp. 01–06. https://doi.org/10.1109/mmsp55362.2022.9950001.

Chicago

Liu, Bochao, Jianghu Lu, Pengju Wang, Junjie Zhang, Dan Zeng, Zhenxing Qian, and Shiming Ge. 2022. “Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation.” IEEE International Workshop on Multimedia Signal Processing (MMSP): 01–06. https://doi.org/10.1109/mmsp55362.2022.9950001.

Harvard

Liu, B. et al. (2022) ‘Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation’, IEEE International Workshop on Multimedia Signal Processing (MMSP), pp. 01–06. doi:10.1109/mmsp55362.2022.9950001.

IEEE

B. Liu et al., “Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation,” IEEE International Workshop on Multimedia Signal Processing (MMSP), pp. 01–06, 2022, doi: 10.1109/mmsp55362.2022.9950001.

BibTeX

@inproceedings{liu2022privacy,
  title     = {{Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation}},
  author    = {Liu, Bochao and Lu, Jianghu and Wang, Pengju and Zhang, Junjie and Zeng, Dan and Qian, Zhenxing and Ge, Shiming},
  booktitle = {IEEE International Workshop on Multimedia Signal Processing (MMSP)},
  year      = {2022},
  pages     = {01--06},
  doi       = {10.1109/mmsp55362.2022.9950001},
  url       = {https://doi.org/10.1109/mmsp55362.2022.9950001}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202311
202411
202522
202611

Its field-weighted citation impact is 0.27: it has been cited 0.27 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • differential privacy
  • teacher-student learning
  • synthetic data generation
  • label privacy
  • privacy-preserving deep learning
  • privacy leakage

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Liu, B., Lu, J., Wang, P., Zhang, J., Zeng, D., Qian, Z., & Ge, S. (2022). Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation. IEEE International Workshop on Multimedia Signal Processing (MMSP), 01–06. https://doi.org/10.1109/mmsp55362.2022.9950001

How many times has this paper been cited?

5 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Bochao Liu, Jianghu Lu, Pengju Wang, Junjie Zhang, Dan Zeng, Zhenxing Qian, Shiming Ge.

What is the DOI of this paper?

10.1109/mmsp55362.2022.9950001, permanently at https://doi.org/10.1109/mmsp55362.2022.9950001. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.