Preprint · 2013 · Open access

Do Deep Nets Really Need to be Deep?

Jimmy Lei Ba, Rich Caruana

arXiv (Cornell University) · Published 21 December 2013 · DOI 10.48550/arxiv.1312.6184

Open PDFView at the publisherCite this paper

In brief

“Do Deep Nets Really Need to be Deep?” is a 2013 preprint by Jimmy Lei Ba and Rich Caruana, published in arXiv (Cornell University). OpenAlex counts 1,743 citations as of 22 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
1,743
Most cited in
2021 (291)
References
23
Authors
2

Abstract

Currently, deep neural networks are the state of the art on problems such as speech recognition and computer vision. In this extended abstract, we show that shallow feed-forward networks can learn the complex functions previously learned by deep nets and achieve accuracies previously only achievable with deep models. Moreover, in some cases the shallow neural nets can learn these deep functions using a total number of parameters similar to the original deep model. We evaluate our method on the TIMIT phoneme recognition task and are able to train shallow fully-connected nets that perform similarly to complex, well-engineered, deep convolutional architectures. Our success in training shallow neural nets to mimic deeper models suggests that there probably exist better algorithms for training shallow feed-forward nets than those currently available.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Ba, J. L., & Caruana, R. (2013). Do Deep Nets Really Need to be Deep? arXiv (Cornell University). https://doi.org/10.48550/arxiv.1312.6184

MLA 9

Ba, Jimmy Lei, and Rich Caruana. “Do Deep Nets Really Need to be Deep?” arXiv (Cornell University), 2013. https://doi.org/10.48550/arxiv.1312.6184.

Chicago

Ba, Jimmy Lei and Rich Caruana. 2013. “Do Deep Nets Really Need to be Deep?” arXiv (Cornell University). https://doi.org/10.48550/arxiv.1312.6184.

Harvard

Ba, J. L. and Caruana, R. (2013) ‘Do Deep Nets Really Need to be Deep?’, arXiv (Cornell University). doi:10.48550/arxiv.1312.6184.

IEEE

J. L. Ba and R. Caruana, “Do Deep Nets Really Need to be Deep?,” arXiv (Cornell University), 2013, doi: 10.48550/arxiv.1312.6184.

BibTeX

@misc{ba2013deep,
  title     = {{Do Deep Nets Really Need to be Deep?}},
  author    = {Ba, Jimmy Lei and Caruana, Rich},
  howpublished = {arXiv (Cornell University)},
  year      = {2013},
  doi       = {10.48550/arxiv.1312.6184},
  url       = {https://doi.org/10.48550/arxiv.1312.6184}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
2017124124
2018167167
2019241241
2020259259
2021291291
2022159159
2023164164
2024115115
20257676
20263131

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications

  • shallow neural networks
  • deep neural network
  • feedforward neural networks
  • network depth
  • speech recognition
  • parameter efficiency

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Ba, J. L., & Caruana, R. (2013). Do Deep Nets Really Need to be Deep? arXiv (Cornell University). https://doi.org/10.48550/arxiv.1312.6184

How many times has this paper been cited?

1,743 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Jimmy Lei Ba, Rich Caruana.

What is the DOI of this paper?

10.48550/arxiv.1312.6184, permanently at https://doi.org/10.48550/arxiv.1312.6184. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.