Preprint · 2018 · Open access
Distributed Stochastic Optimization via Adaptive SGD
arXiv (Cornell University), 31, 1910–1919 · Published 16 February 2018 · DOI 10.48550/arxiv.1802.05811
In brief
“Distributed Stochastic Optimization via Adaptive SGD” is a 2018 preprint by Ashok Cutkosky and Róbert Busa‐Fekete, published in arXiv (Cornell University). OpenAlex counts 20 citations as of 16 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 20
- Most cited in
- 2019 (8)
- References
- 18
- Authors
- 2
Abstract
Stochastic convex optimization algorithms are the most popular way to train machine learning models on large-scale data. Scaling up the training process of these models is crucial, but the most popular algorithm, Stochastic Gradient Descent (SGD), is a serial method that is surprisingly hard to parallelize. In this paper, we propose an efficient distributed stochastic optimization method by combining adaptivity with variance reduction techniques. Our analysis yields a linear speedup in the number of machines, constant memory footprint, and only a logarithmic number of communication rounds. Critically, our approach is a black-box reduction that parallelizes any serial online learning algorithm, streamlining prior analysis and allowing us to leverage the significant progress that has been made in designing adaptive algorithms. In particular, we achieve optimal convergence rates without any prior knowledge of smoothness parameters, yielding a more robust algorithm that reduces the need for hyperparameter tuning. We implement our algorithm in the Spark distributed framework and exhibit dramatic performance gains on large-scale logistic regression problems.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Cutkosky, A., & Busa‐Fekete, R. (2018). Distributed Stochastic Optimization via Adaptive SGD. arXiv (Cornell University), 31, 1910–1919. https://doi.org/10.48550/arxiv.1802.05811
MLA 9
Cutkosky, Ashok, and Róbert Busa‐Fekete. “Distributed Stochastic Optimization via Adaptive SGD.” arXiv (Cornell University), vol. 31, 2018, pp. 1910–1919. https://doi.org/10.48550/arxiv.1802.05811.
Chicago
Cutkosky, Ashok and Róbert Busa‐Fekete. 2018. “Distributed Stochastic Optimization via Adaptive SGD.” arXiv (Cornell University) 31: 1910–1919. https://doi.org/10.48550/arxiv.1802.05811.
Harvard
Cutkosky, A. and Busa‐Fekete, R. (2018) ‘Distributed Stochastic Optimization via Adaptive SGD’, arXiv (Cornell University), 31, pp. 1910–1919. doi:10.48550/arxiv.1802.05811.
IEEE
A. Cutkosky and R. Busa‐Fekete, “Distributed Stochastic Optimization via Adaptive SGD,” arXiv (Cornell University), vol. 31, pp. 1910–1919, 2018, doi: 10.48550/arxiv.1802.05811.
BibTeX
@misc{cutkosky2018distributed,
title = {{Distributed Stochastic Optimization via Adaptive SGD}},
author = {Cutkosky, Ashok and Busa‐Fekete, Róbert},
howpublished = {arXiv (Cornell University)},
year = {2018},
volume = {31},
pages = {1910--1919},
doi = {10.48550/arxiv.1802.05811},
url = {https://doi.org/10.48550/arxiv.1802.05811}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2019 | 8 | |
| 2020 | 4 | |
| 2021 | 3 | |
| 2022 | 2 | |
| 2024 | 2 | |
| 2026 | 1 |
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- distributed stochastic optimization
- variance reduction
- convex optimization
- black-box reductions
- convergence rate
- memory-efficient algorithms
- communication rounds
- Spark
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Cutkosky, A., & Busa‐Fekete, R. (2018). Distributed Stochastic Optimization via Adaptive SGD. arXiv (Cornell University), 31, 1910–1919. https://doi.org/10.48550/arxiv.1802.05811
How many times has this paper been cited?
20 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at arXiv (Cornell University).
Who wrote this paper?
Ashok Cutkosky, Róbert Busa‐Fekete.
What is the DOI of this paper?
10.48550/arxiv.1802.05811. The permanent link is https://doi.org/10.48550/arxiv.1802.05811.
Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.