Preprint · 2017 · Open access
Deep Neural Networks as Gaussian Processes
arXiv (Cornell University) · Published 1 November 2017 · DOI 10.48550/arxiv.1711.00165
In brief
“Deep Neural Networks as Gaussian Processes” is a 2017 preprint by Jaehoon Lee and 5 co-authors, published in arXiv (Cornell University). OpenAlex counts 1,085 citations as of 18 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 1,085
- Most cited in
- 2021 (176)
- References
- 22
- Authors
- 6
Abstract
It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width. This correspondence enables exact Bayesian inference for infinite width neural networks on regression tasks by means of evaluating the corresponding GP. Recently, kernel functions which mimic multi-layer random neural networks have been developed, but only outside of a Bayesian framework. As such, previous work has not identified that these kernels can be used as covariance functions for GPs and allow fully Bayesian prediction with a deep neural network. In this work, we derive the exact equivalence between infinitely wide deep networks and GPs. We further develop a computationally efficient pipeline to compute the covariance function for these GPs. We then use the resulting GPs to perform Bayesian inference for wide deep neural networks on MNIST and CIFAR-10. We observe that trained neural network accuracy approaches that of the corresponding GP with increasing layer width, and that the GP uncertainty is strongly correlated with trained network prediction error. We further find that test performance increases as finite-width trained networks are made wider and more similar to a GP, and thus that GP predictions typically outperform those of finite-width networks. Finally we connect the performance of these GPs to the recent theory of signal propagation in random neural networks.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Lee, J., Bahri, Y., Roman, N., Schoenholz, S. S., Pennington, J., & Sohl‐Dickstein, J. (2017). Deep Neural Networks as Gaussian Processes. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1711.00165
MLA 9
Lee, Jaehoon, et al. “Deep Neural Networks as Gaussian Processes.” arXiv (Cornell University), 2017. https://doi.org/10.48550/arxiv.1711.00165.
Chicago
Lee, Jaehoon, Yasaman Bahri, Novak, Roman, Samuel S. Schoenholz, Jeffrey Pennington, and Jascha Sohl‐Dickstein. 2017. “Deep Neural Networks as Gaussian Processes.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.1711.00165.
Harvard
Lee, J. et al. (2017) ‘Deep Neural Networks as Gaussian Processes’, arXiv (Cornell University). doi:10.48550/arxiv.1711.00165.
IEEE
J. Lee, Y. Bahri, N. Roman, S. S. Schoenholz, J. Pennington, and J. Sohl‐Dickstein, “Deep Neural Networks as Gaussian Processes,” arXiv (Cornell University), 2017, doi: 10.48550/arxiv.1711.00165.
BibTeX
@misc{lee2017deep,
title = {{Deep Neural Networks as Gaussian Processes}},
author = {Lee, Jaehoon and Bahri, Yasaman and Roman, Novak, and Schoenholz, Samuel S. and Pennington, Jeffrey and Sohl‐Dickstein, Jascha},
howpublished = {arXiv (Cornell University)},
year = {2017},
doi = {10.48550/arxiv.1711.00165},
url = {https://doi.org/10.48550/arxiv.1711.00165}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2017 | 2 | |
| 2018 | 39 | |
| 2019 | 97 | |
| 2020 | 163 | |
| 2021 | 176 | |
| 2022 | 128 | |
| 2023 | 124 | |
| 2024 | 140 | |
| 2025 | 143 | |
| 2026 | 73 |
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- deep neural network
- Gaussian processes
- infinite-width neural networks
- Bayesian inference
- covariance function
- kernel functions
- random neural networks
- signal propagation
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Lee, J., Bahri, Y., Roman, N., Schoenholz, S. S., Pennington, J., & Sohl‐Dickstein, J. (2017). Deep Neural Networks as Gaussian Processes. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1711.00165
How many times has this paper been cited?
1,085 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Jaehoon Lee, Yasaman Bahri, Novak, Roman, Samuel S. Schoenholz, Jeffrey Pennington, Jascha Sohl‐Dickstein.
What is the DOI of this paper?
10.48550/arxiv.1711.00165, permanently at https://doi.org/10.48550/arxiv.1711.00165. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.