Conference paper · 2021 · Open access

Activation-level uncertainty in deep neural networks

Pablo Morales-Álvarez, Daniel Hernández-Lobato, Rafael Molina, José Miguel Hernández-Lobato

International Conference on Learning Representations · Published 3 May 2021

Open full textCite this paper

In brief

“Activation-level uncertainty in deep neural networks” is a 2021 conference paper by Pablo Morales-Álvarez and 3 co-authors, published in International Conference on Learning Representations. OpenAlex counts 24 citations as of 18 November 2026, which puts it in the top 6% of works published in 2021 by citations. It is open access: the full text is free to read at International Conference on Learning Representations.

Citations
24
Most cited in
2025 (6)
Field-weighted impact
1.08
References
55
Authors
4

Abstract

Current approaches for uncertainty estimation in deep learning often produce too confident results. Bayesian Neural Networks (BNNs) model uncertainty in the space of weights, which is usually high-dimensional and limits the quality of variational approximations. The more recent functional BNNs (fBNNs) address this only partially because, although the prior is specified in the space of functions, the posterior approximation is still defined in terms of stochastic weights. In this work we propose to move uncertainty from the weights (which are deterministic) to the activation function. Specifically, the activations are modelled with simple 1D Gaussian Processes (GP), for which a triangular kernel inspired by the ReLu non-linearity is explored. Our experiments show that activation-level stochasticity provides more reliable uncertainty estimates than BNN and fBNN, whereas it performs competitively in standard prediction tasks. We also study the connection with deep GPs, both theoretically and empirically. More precisely, we show that activation-level uncertainty requires fewer inducing points and is better suited for deep architectures.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Morales-Álvarez, P., Hernández-Lobato, D., Molina, R., & Hernández-Lobato, J. M. (2021). Activation-level uncertainty in deep neural networks. International Conference on Learning Representations. https://openreview.net/pdf?id=UvBPbpvHRj-

MLA 9

Morales-Álvarez, Pablo, et al. “Activation-level uncertainty in deep neural networks.” International Conference on Learning Representations, 2021. https://openreview.net/pdf?id=UvBPbpvHRj-.

Chicago

Morales-Álvarez, Pablo, Daniel Hernández-Lobato, Rafael Molina, and José Miguel Hernández-Lobato. 2021. “Activation-level uncertainty in deep neural networks.” International Conference on Learning Representations. https://openreview.net/pdf?id=UvBPbpvHRj-.

Harvard

Morales-Álvarez, P. et al. (2021) ‘Activation-level uncertainty in deep neural networks’, International Conference on Learning Representations. Available at: https://openreview.net/pdf?id=UvBPbpvHRj-.

IEEE

P. Morales-Álvarez, D. Hernández-Lobato, R. Molina, and J. M. Hernández-Lobato, “Activation-level uncertainty in deep neural networks,” International Conference on Learning Representations, 2021.

BibTeX

@inproceedings{moralesalvarez2021activation,
  title     = {{Activation-level uncertainty in deep neural networks}},
  author    = {Morales-Álvarez, Pablo and Hernández-Lobato, Daniel and Molina, Rafael and Hernández-Lobato, José Miguel},
  booktitle = {International Conference on Learning Representations},
  year      = {2021},
  url       = {https://openreview.net/pdf?id=UvBPbpvHRj-}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202144
202233
202333
202455
202566
202633

Its field-weighted citation impact is 1.08: it has been cited 1.08 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference

  • uncertainty estimation
  • deep neural network
  • Bayesian neural networks
  • Gaussian processes
  • triangular kernel
  • deep Gaussian processes
  • inducing points
  • overconfidence

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Morales-Álvarez, P., Hernández-Lobato, D., Molina, R., & Hernández-Lobato, J. M. (2021). Activation-level uncertainty in deep neural networks. International Conference on Learning Representations. https://openreview.net/pdf?id=UvBPbpvHRj-

How many times has this paper been cited?

24 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at International Conference on Learning Representations: sign in and open the full text from this page.

Who wrote this paper?

Pablo Morales-Álvarez, Daniel Hernández-Lobato, Rafael Molina, José Miguel Hernández-Lobato.

Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.