Conference paper · 2021 · Open access
Activation-level uncertainty in deep neural networks
International Conference on Learning Representations · Published 3 May 2021
In brief
“Activation-level uncertainty in deep neural networks” is a 2021 conference paper by Pablo Morales-Álvarez and 3 co-authors, published in International Conference on Learning Representations. OpenAlex counts 24 citations as of 18 November 2026, which puts it in the top 6% of works published in 2021 by citations. It is open access: the full text is free to read at International Conference on Learning Representations.
- Citations
- 24
- Most cited in
- 2025 (6)
- Field-weighted impact
- 1.08
- References
- 55
- Authors
- 4
Abstract
Current approaches for uncertainty estimation in deep learning often produce too confident results. Bayesian Neural Networks (BNNs) model uncertainty in the space of weights, which is usually high-dimensional and limits the quality of variational approximations. The more recent functional BNNs (fBNNs) address this only partially because, although the prior is specified in the space of functions, the posterior approximation is still defined in terms of stochastic weights. In this work we propose to move uncertainty from the weights (which are deterministic) to the activation function. Specifically, the activations are modelled with simple 1D Gaussian Processes (GP), for which a triangular kernel inspired by the ReLu non-linearity is explored. Our experiments show that activation-level stochasticity provides more reliable uncertainty estimates than BNN and fBNN, whereas it performs competitively in standard prediction tasks. We also study the connection with deep GPs, both theoretically and empirically. More precisely, we show that activation-level uncertainty requires fewer inducing points and is better suited for deep architectures.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Morales-Álvarez, P., Hernández-Lobato, D., Molina, R., & Hernández-Lobato, J. M. (2021). Activation-level uncertainty in deep neural networks. International Conference on Learning Representations. https://openreview.net/pdf?id=UvBPbpvHRj-
MLA 9
Morales-Álvarez, Pablo, et al. “Activation-level uncertainty in deep neural networks.” International Conference on Learning Representations, 2021. https://openreview.net/pdf?id=UvBPbpvHRj-.
Chicago
Morales-Álvarez, Pablo, Daniel Hernández-Lobato, Rafael Molina, and José Miguel Hernández-Lobato. 2021. “Activation-level uncertainty in deep neural networks.” International Conference on Learning Representations. https://openreview.net/pdf?id=UvBPbpvHRj-.
Harvard
Morales-Álvarez, P. et al. (2021) ‘Activation-level uncertainty in deep neural networks’, International Conference on Learning Representations. Available at: https://openreview.net/pdf?id=UvBPbpvHRj-.
IEEE
P. Morales-Álvarez, D. Hernández-Lobato, R. Molina, and J. M. Hernández-Lobato, “Activation-level uncertainty in deep neural networks,” International Conference on Learning Representations, 2021.
BibTeX
@inproceedings{moralesalvarez2021activation,
title = {{Activation-level uncertainty in deep neural networks}},
author = {Morales-Álvarez, Pablo and Hernández-Lobato, Daniel and Molina, Rafael and Hernández-Lobato, José Miguel},
booktitle = {International Conference on Learning Representations},
year = {2021},
url = {https://openreview.net/pdf?id=UvBPbpvHRj-}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2021 | 4 | |
| 2022 | 3 | |
| 2023 | 3 | |
| 2024 | 5 | |
| 2025 | 6 | |
| 2026 | 3 |
Its field-weighted citation impact is 1.08: it has been cited 1.08 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- uncertainty estimation
- deep neural network
- Bayesian neural networks
- Gaussian processes
- triangular kernel
- deep Gaussian processes
- inducing points
- overconfidence
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Morales-Álvarez, P., Hernández-Lobato, D., Molina, R., & Hernández-Lobato, J. M. (2021). Activation-level uncertainty in deep neural networks. International Conference on Learning Representations. https://openreview.net/pdf?id=UvBPbpvHRj-
How many times has this paper been cited?
24 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at International Conference on Learning Representations: sign in and open the full text from this page.
Who wrote this paper?
Pablo Morales-Álvarez, Daniel Hernández-Lobato, Rafael Molina, José Miguel Hernández-Lobato.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.