Preprint · 2016 · Open access

Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, Kunal Talwar

arXiv (Cornell University) · Published 18 October 2016 · DOI 10.48550/arxiv.1610.05755

Open PDFView at the publisherCite this paper

In brief

“Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data” is a 2016 preprint by Nicolas Papernot and 4 co-authors, published in arXiv (Cornell University). OpenAlex counts 1,249 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
1,249
Most cited in
2023 (187)
References
43
Authors
5

Abstract

Some machine learning applications involve training data that is sensitive, such as the medical histories of patients in a clinical trial. A model may inadvertently and implicitly store some of its training data; careful analysis of the model may therefore reveal sensitive information. To address this problem, we demonstrate a generally applicable approach to providing strong privacy guarantees for training data: Private Aggregation of Teacher Ensembles (PATE). The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users. Because they rely directly on sensitive data, these models are not published, but instead used as "teachers" for a "student" model. The student learns to predict an output chosen by noisy voting among all of the teachers, and cannot directly access an individual teacher or the underlying data or parameters. The student's privacy properties can be understood both intuitively (since no single teacher and thus no single dataset dictates the student's training) and formally, in terms of differential privacy. These properties hold even if an adversary can not only query the student but also inspect its internal workings. Compared with previous work, the approach imposes only weak assumptions on how teachers are trained: it applies to any model, including non-convex models like DNNs. We achieve state-of-the-art privacy/utility trade-offs on MNIST and SVHN thanks to an improved privacy analysis and semi-supervised learning.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I., & Talwar, K. (2016). Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1610.05755

MLA 9

Papernot, Nicolas, et al. “Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data.” arXiv (Cornell University), 2016. https://doi.org/10.48550/arxiv.1610.05755.

Chicago

Papernot, Nicolas, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, and Kunal Talwar. 2016. “Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.1610.05755.

Harvard

Papernot, N. et al. (2016) ‘Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data’, arXiv (Cornell University). doi:10.48550/arxiv.1610.05755.

IEEE

N. Papernot, M. Abadi, Ú. Erlingsson, I. Goodfellow, and K. Talwar, “Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data,” arXiv (Cornell University), 2016, doi: 10.48550/arxiv.1610.05755.

BibTeX

@misc{papernot2016semi,
  title     = {{Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data}},
  author    = {Papernot, Nicolas and Abadi, Martín and Erlingsson, Úlfar and Goodfellow, Ian and Talwar, Kunal},
  howpublished = {arXiv (Cornell University)},
  year      = {2016},
  doi       = {10.48550/arxiv.1610.05755},
  url       = {https://doi.org/10.48550/arxiv.1610.05755}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
20172525
20186363
2019119119
2020139139
2021156156
2022139139
2023187187
2024162162
2025139139
2026117117

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • pâté
  • differential privacy
  • semi-supervised learning
  • training data privacy
  • knowledge transfer
  • deep neural network
  • privacy-utility trade-off

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I., & Talwar, K. (2016). Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1610.05755

How many times has this paper been cited?

1,249 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, Kunal Talwar.

What is the DOI of this paper?

10.48550/arxiv.1610.05755, permanently at https://doi.org/10.48550/arxiv.1610.05755. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.