Preprint · 2016 · Open access
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
arXiv (Cornell University) · Published 18 October 2016 · DOI 10.48550/arxiv.1610.05755
In brief
“Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data” is a 2016 preprint by Nicolas Papernot and 4 co-authors, published in arXiv (Cornell University). OpenAlex counts 1,249 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 1,249
- Most cited in
- 2023 (187)
- References
- 43
- Authors
- 5
Abstract
Some machine learning applications involve training data that is sensitive, such as the medical histories of patients in a clinical trial. A model may inadvertently and implicitly store some of its training data; careful analysis of the model may therefore reveal sensitive information. To address this problem, we demonstrate a generally applicable approach to providing strong privacy guarantees for training data: Private Aggregation of Teacher Ensembles (PATE). The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users. Because they rely directly on sensitive data, these models are not published, but instead used as "teachers" for a "student" model. The student learns to predict an output chosen by noisy voting among all of the teachers, and cannot directly access an individual teacher or the underlying data or parameters. The student's privacy properties can be understood both intuitively (since no single teacher and thus no single dataset dictates the student's training) and formally, in terms of differential privacy. These properties hold even if an adversary can not only query the student but also inspect its internal workings. Compared with previous work, the approach imposes only weak assumptions on how teachers are trained: it applies to any model, including non-convex models like DNNs. We achieve state-of-the-art privacy/utility trade-offs on MNIST and SVHN thanks to an improved privacy analysis and semi-supervised learning.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I., & Talwar, K. (2016). Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1610.05755
MLA 9
Papernot, Nicolas, et al. “Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data.” arXiv (Cornell University), 2016. https://doi.org/10.48550/arxiv.1610.05755.
Chicago
Papernot, Nicolas, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, and Kunal Talwar. 2016. “Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.1610.05755.
Harvard
Papernot, N. et al. (2016) ‘Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data’, arXiv (Cornell University). doi:10.48550/arxiv.1610.05755.
IEEE
N. Papernot, M. Abadi, Ú. Erlingsson, I. Goodfellow, and K. Talwar, “Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data,” arXiv (Cornell University), 2016, doi: 10.48550/arxiv.1610.05755.
BibTeX
@misc{papernot2016semi,
title = {{Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data}},
author = {Papernot, Nicolas and Abadi, Martín and Erlingsson, Úlfar and Goodfellow, Ian and Talwar, Kunal},
howpublished = {arXiv (Cornell University)},
year = {2016},
doi = {10.48550/arxiv.1610.05755},
url = {https://doi.org/10.48550/arxiv.1610.05755}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2017 | 25 | |
| 2018 | 63 | |
| 2019 | 119 | |
| 2020 | 139 | |
| 2021 | 156 | |
| 2022 | 139 | |
| 2023 | 187 | |
| 2024 | 162 | |
| 2025 | 139 | |
| 2026 | 117 |
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- pâté
- differential privacy
- semi-supervised learning
- training data privacy
- knowledge transfer
- deep neural network
- privacy-utility trade-off
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I., & Talwar, K. (2016). Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1610.05755
How many times has this paper been cited?
1,249 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, Kunal Talwar.
What is the DOI of this paper?
10.48550/arxiv.1610.05755, permanently at https://doi.org/10.48550/arxiv.1610.05755. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.