Preprint · 2018 · Open access
Hyperbolic Attention Networks
arXiv (Cornell University) · Published 24 May 2018 · DOI 10.48550/arxiv.1805.09786
In brief
“Hyperbolic Attention Networks” is a 2018 preprint by Çağlar Gülçehre and 10 co-authors, published in arXiv (Cornell University). OpenAlex counts 262 citations as of 20 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 262
- Most cited in
- 2021 (43)
- References
- 48
- Authors
- 11
Abstract
We introduce hyperbolic attention networks to endow neural networks with enough capacity to match the complexity of data with hierarchical and power-law structure. A few recent approaches have successfully demonstrated the benefits of imposing hyperbolic geometry on the parameters of shallow networks. We extend this line of work by imposing hyperbolic geometry on the activations of neural networks. This allows us to exploit hyperbolic geometry to reason about embeddings produced by deep networks. We achieve this by re-expressing the ubiquitous mechanism of soft attention in terms of operations defined for hyperboloid and Klein models. Our method shows improvements in terms of generalization on neural machine translation, learning on graphs and visual question answering tasks while keeping the neural representations compact.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Gülçehre, Ç., Denil, M., Malinowski, M., Razavi, A. S., Pascanu, R., Hermann, K. M., Battaglia, P., Bapst, V., Raposo, D. C., Santoro, A., & de Freitas, N. (2018). Hyperbolic Attention Networks. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1805.09786
MLA 9
Gülçehre, Çağlar, et al. “Hyperbolic Attention Networks.” arXiv (Cornell University), 2018. https://doi.org/10.48550/arxiv.1805.09786.
Chicago
Gülçehre, Çağlar, Misha Denil, Mateusz Malinowski, Ali Shervin Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, et al. 2018. “Hyperbolic Attention Networks.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.1805.09786.
Harvard
Gülçehre, Ç. et al. (2018) ‘Hyperbolic Attention Networks’, arXiv (Cornell University). doi:10.48550/arxiv.1805.09786.
IEEE
Ç. Gülçehre et al., “Hyperbolic Attention Networks,” arXiv (Cornell University), 2018, doi: 10.48550/arxiv.1805.09786.
BibTeX
@misc{gulcehre2018hyperbolic,
title = {{Hyperbolic Attention Networks}},
author = {Gülçehre, Çağlar and Denil, Misha and Malinowski, Mateusz and Razavi, Ali Shervin and Pascanu, Razvan and Hermann, Karl Moritz and Battaglia, Peter and Bapst, Victor and Raposo, David Chillón and Santoro, Adam and de Freitas, Nando},
howpublished = {arXiv (Cornell University)},
year = {2018},
doi = {10.48550/arxiv.1805.09786},
url = {https://doi.org/10.48550/arxiv.1805.09786}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2018 | 7 | |
| 2019 | 19 | |
| 2020 | 24 | |
| 2021 | 43 | |
| 2022 | 31 | |
| 2023 | 38 | |
| 2024 | 32 | |
| 2025 | 31 | |
| 2026 | 37 |
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- hyperbolic geometry
- soft attention
- hyperboloid model
- Klein model
- hierarchical data
- neural machine translation
- graph learning
- visual question answering
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Gülçehre, Ç., Denil, M., Malinowski, M., Razavi, A. S., Pascanu, R., Hermann, K. M., Battaglia, P., Bapst, V., Raposo, D. C., Santoro, A., & de Freitas, N. (2018). Hyperbolic Attention Networks. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1805.09786
How many times has this paper been cited?
262 times, according to OpenAlex on 20 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Çağlar Gülçehre, Misha Denil, Mateusz Malinowski, Ali Shervin Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Chillón Raposo, Adam Santoro, Nando de Freitas.
What is the DOI of this paper?
10.48550/arxiv.1805.09786, permanently at https://doi.org/10.48550/arxiv.1805.09786. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 20 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.