Journal article · 2021 · Open access
Entropy and Relative Entropy From Information-Theoretic Principles
IEEE Transactions on Information Theory, 67(10), 6313–6327 · Published 14 May 2021 · DOI 10.1109/tit.2021.3078337
In brief
“Entropy and Relative Entropy From Information-Theoretic Principles” is a 2021 journal article by Gilad Gour and Marco Tomamichel, published in IEEE Transactions on Information Theory. OpenAlex counts 40 citations as of 15 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 40
- Most cited in
- 2025 (9)
- Field-weighted impact
- 1.88
- References
- 28
- Authors
- 2
Abstract
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find that these axioms induce sufficient structure to establish continuity in the interior of the probability simplex and meaningful upper and lower bounds, e.g., we find that every relative entropy satisfying these axioms must lie between the Rényi divergences of order 0 and ∞. We further show simple conditions for positive definiteness of such relative entropies and a characterisation in terms of a variant of relative trumping. Our main result is a one-to-one correspondence between entropies and relative entropies.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Gour, G., & Tomamichel, M. (2021). Entropy and Relative Entropy From Information-Theoretic Principles. IEEE Transactions on Information Theory, 67(10), 6313–6327. https://doi.org/10.1109/tit.2021.3078337
MLA 9
Gour, Gilad, and Marco Tomamichel. “Entropy and Relative Entropy From Information-Theoretic Principles.” IEEE Transactions on Information Theory, vol. 67, no. 10, 2021, pp. 6313–6327. https://doi.org/10.1109/tit.2021.3078337.
Chicago
Gour, Gilad and Marco Tomamichel. 2021. “Entropy and Relative Entropy From Information-Theoretic Principles.” IEEE Transactions on Information Theory 67 (10): 6313–6327. https://doi.org/10.1109/tit.2021.3078337.
Harvard
Gour, G. and Tomamichel, M. (2021) ‘Entropy and Relative Entropy From Information-Theoretic Principles’, IEEE Transactions on Information Theory, 67(10), pp. 6313–6327. doi:10.1109/tit.2021.3078337.
IEEE
G. Gour and M. Tomamichel, “Entropy and Relative Entropy From Information-Theoretic Principles,” IEEE Transactions on Information Theory, vol. 67, no. 10, pp. 6313–6327, 2021, doi: 10.1109/tit.2021.3078337.
BibTeX
@article{gour2021entropy,
title = {{Entropy and Relative Entropy From Information-Theoretic Principles}},
author = {Gour, Gilad and Tomamichel, Marco},
journal = {IEEE Transactions on Information Theory},
year = {2021},
volume = {67},
number = {10},
pages = {6313--6327},
doi = {10.1109/tit.2021.3078337},
url = {https://doi.org/10.1109/tit.2021.3078337}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2020 | 1 | |
| 2021 | 3 | |
| 2022 | 7 | |
| 2023 | 4 | |
| 2024 | 8 | |
| 2025 | 9 | |
| 2026 | 8 |
Its field-weighted citation impact is 1.88: it has been cited 1.88 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Statistical Mechanics and Entropy
- entropy (information theory)
- Kullback-Leibler divergence
- Rényi divergence
- data processing inequality
- positive definiteness
- probability simplex
Work with this paper
Most-cited papers in Physics and Astronomy
- Generalized Gradient Approximation Made SimpleJohn P. Perdew, Kieron Burke, Matthias Ernzerhof · Physical Review Letters · 1996 · 217,345 citations
- Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis setGeorg Kresse, Jürgen Furthmüller · Physical review. B, Condensed matter · 1996 · 125,164 citations
- Density-functional thermochemistry. III. The role of exact exchangeAxel D. Becke · The Journal of Chemical Physics · 1993 · 104,772 citations
- Development of the Colle-Salvetti correlation-energy formula into a functional of the electron densityChengteh Lee, Weitao Yang, Robert G. Parr · Physical review. B, Condensed matter · 1988 · 101,730 citations
- Projector augmented-wave methodPeter E. Blöchl · Physical review. B, Condensed matter · 1994 · 93,520 citations
Questions about this paper
How do I cite this paper in APA style?
Gour, G., & Tomamichel, M. (2021). Entropy and Relative Entropy From Information-Theoretic Principles. IEEE Transactions on Information Theory, 67(10), 6313–6327. https://doi.org/10.1109/tit.2021.3078337
How many times has this paper been cited?
40 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at arXiv (Cornell University).
Who wrote this paper?
Gilad Gour, Marco Tomamichel.
What is the DOI of this paper?
10.1109/tit.2021.3078337. The permanent link is https://doi.org/10.1109/tit.2021.3078337.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.