Journal article · 2026 · Open access
Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective
SIAM Review, 68(2), 293–345 · Published 13 May 2026 · DOI 10.1137/24m1715519
In brief
“Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective” is a 2026 journal article by Joel Wendin and Claudio Altafini, published in SIAM Review. OpenAlex counts 4 citations as of 17 November 2026, which puts it in the top 3% of works published in 2026 by citations. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 4
- Most cited in
- 2026 (4)
- Field-weighted impact
- 4.33
- References
- 29
- Authors
- 2
Abstract
This paper surveys recent progress in understanding the dynamics and loss landscape of the gradient flow equations associated with deep linear neural networks, i.e., the gradient descent training dynamics (in the limit when the step size goes to 0) of deep neural networks missing the activation functions and subject to quadratic loss functions. When formulated in terms of the adjacency matrix of the neural network, as is done in this paper, these gradient flow equations form a class of converging matrix ODEs which is nilpotent, polynomial, isospectral, and with conservation laws. A detailed description of the loss landscape shows that it is described in detail and is characterized by infinitely-many global minima and saddle points, both strict and nonstrict, but that it lacks local minima and maxima. The loss function itself is a positive semidefinite Lyapunov function for the gradient flow, and its level sets are unbounded invariant sets of critical points with critical values that correspond to the amount of singular values of the input-output data learnt by the gradient along a certain trajectory. The adjacency matrix representation we use in the paper allows us to highlight the existence of a quotient space structure in which each critical value of the loss function is represented only once, while all other critical points with the same critical value belong to the fiber associated to the quotient space. It also allows us to easily determine stable and unstable submanifolds at the saddle points, even when the Hessian fails to obtain them.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Wendin, J., & Altafini, C. (2026). Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective. SIAM Review, 68(2), 293–345. https://doi.org/10.1137/24m1715519
MLA 9
Wendin, Joel, and Claudio Altafini. “Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective.” SIAM Review, vol. 68, no. 2, 2026, pp. 293–345. https://doi.org/10.1137/24m1715519.
Chicago
Wendin, Joel and Claudio Altafini. 2026. “Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective.” SIAM Review 68 (2): 293–345. https://doi.org/10.1137/24m1715519.
Harvard
Wendin, J. and Altafini, C. (2026) ‘Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective’, SIAM Review, 68(2), pp. 293–345. doi:10.1137/24m1715519.
IEEE
J. Wendin and C. Altafini, “Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective,” SIAM Review, vol. 68, no. 2, pp. 293–345, 2026, doi: 10.1137/24m1715519.
BibTeX
@article{wendin2026gradient,
title = {{Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective}},
author = {Wendin, Joel and Altafini, Claudio},
journal = {SIAM Review},
year = {2026},
volume = {68},
number = {2},
pages = {293--345},
doi = {10.1137/24m1715519},
url = {https://doi.org/10.1137/24m1715519}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- gradient flow
- loss landscape
- gradient descent dynamics
- conservation laws
- Lyapunov function
- saddle points
- critical points
- quotient spaces
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Wendin, J., & Altafini, C. (2026). Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective. SIAM Review, 68(2), 293–345. https://doi.org/10.1137/24m1715519
How many times has this paper been cited?
4 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Joel Wendin, Claudio Altafini.
What is the DOI of this paper?
10.1137/24m1715519, permanently at https://doi.org/10.1137/24m1715519. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.