Preprint · 2018 · Open access

Distributed Stochastic Optimization via Adaptive SGD

Ashok Cutkosky, Róbert Busa‐Fekete

arXiv (Cornell University), 31, 1910–1919 · Published 16 February 2018 · DOI 10.48550/arxiv.1802.05811

Free PDFView at the publisherCite this paper

In brief

“Distributed Stochastic Optimization via Adaptive SGD” is a 2018 preprint by Ashok Cutkosky and Róbert Busa‐Fekete, published in arXiv (Cornell University). OpenAlex counts 20 citations as of 16 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
20
Most cited in
2019 (8)
References
18
Authors
2

Abstract

Stochastic convex optimization algorithms are the most popular way to train machine learning models on large-scale data. Scaling up the training process of these models is crucial, but the most popular algorithm, Stochastic Gradient Descent (SGD), is a serial method that is surprisingly hard to parallelize. In this paper, we propose an efficient distributed stochastic optimization method by combining adaptivity with variance reduction techniques. Our analysis yields a linear speedup in the number of machines, constant memory footprint, and only a logarithmic number of communication rounds. Critically, our approach is a black-box reduction that parallelizes any serial online learning algorithm, streamlining prior analysis and allowing us to leverage the significant progress that has been made in designing adaptive algorithms. In particular, we achieve optimal convergence rates without any prior knowledge of smoothness parameters, yielding a more robust algorithm that reduces the need for hyperparameter tuning. We implement our algorithm in the Spark distributed framework and exhibit dramatic performance gains on large-scale logistic regression problems.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Cutkosky, A., & Busa‐Fekete, R. (2018). Distributed Stochastic Optimization via Adaptive SGD. arXiv (Cornell University), 31, 1910–1919. https://doi.org/10.48550/arxiv.1802.05811

MLA 9

Cutkosky, Ashok, and Róbert Busa‐Fekete. “Distributed Stochastic Optimization via Adaptive SGD.” arXiv (Cornell University), vol. 31, 2018, pp. 1910–1919. https://doi.org/10.48550/arxiv.1802.05811.

Chicago

Cutkosky, Ashok and Róbert Busa‐Fekete. 2018. “Distributed Stochastic Optimization via Adaptive SGD.” arXiv (Cornell University) 31: 1910–1919. https://doi.org/10.48550/arxiv.1802.05811.

Harvard

Cutkosky, A. and Busa‐Fekete, R. (2018) ‘Distributed Stochastic Optimization via Adaptive SGD’, arXiv (Cornell University), 31, pp. 1910–1919. doi:10.48550/arxiv.1802.05811.

IEEE

A. Cutkosky and R. Busa‐Fekete, “Distributed Stochastic Optimization via Adaptive SGD,” arXiv (Cornell University), vol. 31, pp. 1910–1919, 2018, doi: 10.48550/arxiv.1802.05811.

BibTeX

@misc{cutkosky2018distributed,
  title     = {{Distributed Stochastic Optimization via Adaptive SGD}},
  author    = {Cutkosky, Ashok and Busa‐Fekete, Róbert},
  howpublished = {arXiv (Cornell University)},
  year      = {2018},
  volume    = {31},
  pages     = {1910--1919},
  doi       = {10.48550/arxiv.1802.05811},
  url       = {https://doi.org/10.48550/arxiv.1802.05811}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
201988
202044
202133
202222
202422
202611

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques

  • distributed stochastic optimization
  • variance reduction
  • convex optimization
  • black-box reductions
  • convergence rate
  • memory-efficient algorithms
  • communication rounds
  • Spark

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Cutkosky, A., & Busa‐Fekete, R. (2018). Distributed Stochastic Optimization via Adaptive SGD. arXiv (Cornell University), 31, 1910–1919. https://doi.org/10.48550/arxiv.1802.05811

How many times has this paper been cited?

20 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at arXiv (Cornell University).

Who wrote this paper?

Ashok Cutkosky, Róbert Busa‐Fekete.

What is the DOI of this paper?

10.48550/arxiv.1802.05811. The permanent link is https://doi.org/10.48550/arxiv.1802.05811.

Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.