Preprint · 2018 · Open access

Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes.

Roman Novak, Lechao Xiao, Jaehoon Lee, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl‐Dickstein

arXiv (Cornell University) · Published 11 October 2018

Read the free full textCite this paper

In brief

“Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes.” is a 2018 preprint by Roman Novak and 6 co-authors, published in arXiv (Cornell University). OpenAlex counts 14 citations as of 16 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
14
Most cited in
2023 (4)
Authors
7

Abstract

There is a previously identified equivalence between wide fully connected neural networks (FCNs) and Gaussian processes (GPs). This equivalence enables, for instance, test set predictions that would have resulted from a fully Bayesian, infinitely wide trained FCN to be computed without ever instantiating the FCN, but by instead evaluating the corresponding GP. In this work, we derive an analogous equivalence for multi-layer convolutional neural networks (CNNs) both with and without pooling layers, and achieve state of the art results on CIFAR10 for GPs without trainable kernels. We also introduce a Monte Carlo method to estimate the GP corresponding to a given neural network architecture, even in cases where the analytic form has too many terms to be computationally feasible. Surprisingly, in the absence of pooling layers, the GPs corresponding to CNNs with and without weight sharing are identical. As a consequence, translation equivariance, beneficial in finite channel CNNs trained with stochastic gradient descent (SGD), is guaranteed to play no role in the Bayesian treatment of the infinite channel limit - a qualitative difference between the two regimes that is not present in the FCN case. We confirm experimentally, that while in some scenarios the performance of SGD-trained finite CNNs approaches that of the corresponding GPs as the channel count increases, with careful tuning SGD-trained CNNs can significantly outperform their corresponding GPs, suggesting advantages from SGD training compared to fully Bayesian parameter estimation.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Novak, R., Xiao, L., Lee, J., Bahri, Y., Abolafia, D. A., Pennington, J., & Sohl‐Dickstein, J. (2018). Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes. arXiv (Cornell University). https://arxiv.org/abs/1810.05148v1

MLA 9

Novak, Roman, et al. “Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes.” arXiv (Cornell University), 2018. https://arxiv.org/abs/1810.05148v1.

Chicago

Novak, Roman, Lechao Xiao, Jaehoon Lee, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, and Jascha Sohl‐Dickstein. 2018. “Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes.” arXiv (Cornell University). https://arxiv.org/abs/1810.05148v1.

Harvard

Novak, R. et al. (2018) ‘Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes.’, arXiv (Cornell University). Available at: https://arxiv.org/abs/1810.05148v1.

IEEE

R. Novak et al., “Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes.,” arXiv (Cornell University), 2018.

BibTeX

@misc{novak2018bayesian,
  title     = {{Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes.}},
  author    = {Novak, Roman and Xiao, Lechao and Lee, Jaehoon and Bahri, Yasaman and Abolafia, Daniel A. and Pennington, Jeffrey and Sohl‐Dickstein, Jascha},
  howpublished = {arXiv (Cornell University)},
  year      = {2018},
  url       = {https://arxiv.org/abs/1810.05148v1}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
201822
201911
202033
202111
202211
202344
202411
202511

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference

  • Bayesian convolutional neural network
  • Gaussian processes
  • pooling layers
  • parameter sharing
  • translation equivariance
  • stochastic gradient descent
  • CIFAR-10

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Novak, R., Xiao, L., Lee, J., Bahri, Y., Abolafia, D. A., Pennington, J., & Sohl‐Dickstein, J. (2018). Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes. arXiv (Cornell University). https://arxiv.org/abs/1810.05148v1

How many times has this paper been cited?

14 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at arXiv (Cornell University).

Who wrote this paper?

Roman Novak, Lechao Xiao, Jaehoon Lee, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl‐Dickstein.

Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.