Preprint · 2018 · Open access

ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers

Ao Ren, Tianyun Zhang, Shaokai Ye, Li, Jiayu, Wenyao Xu, Xuehai Qian, Xue Lin, Yanzhi Wang

arXiv (Cornell University) · Published 31 December 2018 · DOI 10.48550/arxiv.1812.11677

Open PDFView at the publisherCite this paper

In brief

“ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers” is a 2018 preprint by Ao Ren and 7 co-authors, published in arXiv (Cornell University). OpenAlex counts 16 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
16
Most cited in
2021 (4)
References
66
Authors
8

Abstract

To facilitate efficient embedded and hardware implementations of deep neural networks (DNNs), two important categories of DNN model compression techniques: weight pruning and weight quantization are investigated. The former leverages the redundancy in the number of weights, whereas the latter leverages the redundancy in bit representation of weights. However, there lacks a systematic framework of joint weight pruning and quantization of DNNs, thereby limiting the available model compression ratio. Moreover, the computation reduction, energy efficiency improvement, and hardware performance overhead need to be accounted for besides simply model size reduction. To address these limitations, we present ADMM-NN, the first algorithm-hardware co-optimization framework of DNNs using Alternating Direction Method of Multipliers (ADMM), a powerful technique to deal with non-convex optimization problems with possibly combinatorial constraints. The first part of ADMM-NN is a systematic, joint framework of DNN weight pruning and quantization using ADMM. It can be understood as a smart regularization technique with regularization target dynamically updated in each ADMM iteration, thereby resulting in higher performance in model compression than prior work. The second part is hardware-aware DNN optimizations to facilitate hardware-level implementations. Without accuracy loss, we can achieve 85$\times$ and 24$\times$ pruning on LeNet-5 and AlexNet models, respectively, significantly higher than prior work. The improvement becomes more significant when focusing on computation reductions. Combining weight pruning and quantization, we achieve 1,910$\times$ and 231$\times$ reductions in overall model size on these two benchmarks, when focusing on data storage. Highly promising results are also observed on other representative DNNs such as VGGNet and ResNet-50.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Ren, A., Zhang, T., Ye, S., Jiayu, L., Xu, W., Qian, X., Lin, X., & Wang, Y. (2018). ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1812.11677

MLA 9

Ren, Ao, et al. “ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers.” arXiv (Cornell University), 2018. https://doi.org/10.48550/arxiv.1812.11677.

Chicago

Ren, Ao, Tianyun Zhang, Shaokai Ye, Li, Jiayu, Wenyao Xu, Xuehai Qian, Xue Lin, and Yanzhi Wang. 2018. “ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.1812.11677.

Harvard

Ren, A. et al. (2018) ‘ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers’, arXiv (Cornell University). doi:10.48550/arxiv.1812.11677.

IEEE

A. Ren et al., “ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers,” arXiv (Cornell University), 2018, doi: 10.48550/arxiv.1812.11677.

BibTeX

@misc{ren2018admm,
  title     = {{ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers}},
  author    = {Ren, Ao and Zhang, Tianyun and Ye, Shaokai and Jiayu, Li, and Xu, Wenyao and Qian, Xuehai and Lin, Xue and Wang, Yanzhi},
  howpublished = {arXiv (Cornell University)},
  year      = {2018},
  doi       = {10.48550/arxiv.1812.11677},
  url       = {https://doi.org/10.48550/arxiv.1812.11677}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
201911
202022
202144
202244
202322
202411
202622

Topic and keywords

Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications

  • algorithm-hardware co-design
  • deep neural network
  • alternating direction method of multipliers
  • weight pruning
  • weight quantization
  • dynamic regularization
  • model compression
  • embedded hardware implementation

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Ren, A., Zhang, T., Ye, S., Jiayu, L., Xu, W., Qian, X., Lin, X., & Wang, Y. (2018). ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1812.11677

How many times has this paper been cited?

16 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Ao Ren, Tianyun Zhang, Shaokai Ye, Li, Jiayu, Wenyao Xu, Xuehai Qian, Xue Lin, Yanzhi Wang.

What is the DOI of this paper?

10.48550/arxiv.1812.11677, permanently at https://doi.org/10.48550/arxiv.1812.11677. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.