Preprint · 2023 · Open access
SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity
arXiv (Cornell University) · Published 30 October 2023 · DOI 10.48550/arxiv.2310.19509
In brief
“SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity” is a 2023 preprint by Haitao Xu and 9 co-authors, published in arXiv (Cornell University). OpenAlex counts 5 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 5
- Most cited in
- 2026 (4)
- References
- 30
- Authors
- 10
Abstract
To address the challenge of increasing network size, researchers have developed sparse models through network pruning. However, maintaining model accuracy while achieving significant speedups on general computing devices remains an open problem. In this paper, we present a novel mobile inference acceleration framework SparseByteNN, which leverages fine-grained kernel sparsity to achieve real-time execution as well as high accuracy. Our framework consists of two parts: (a) A fine-grained kernel sparsity schema with a sparsity granularity between structured pruning and unstructured pruning. It designs multiple sparse patterns for different operators. Combined with our proposed whole network rearrangement strategy, the schema achieves a high compression rate and high precision at the same time. (b) Inference engine co-optimized with the sparse pattern. The conventional wisdom is that this reduction in theoretical FLOPs does not translate into real-world efficiency gains. We aim to correct this misconception by introducing a family of efficient sparse kernels for ARM and WebAssembly. Equipped with our efficient implementation of sparse primitives, we show that sparse versions of MobileNet-v1 outperform strong dense baselines on the efficiency-accuracy curve. Experimental results on Qualcomm 855 show that for 30% sparse MobileNet-v1, SparseByteNN achieves 1.27x speedup over the dense version and 1.29x speedup over the state-of-the-art sparse inference engine MNN with a slight accuracy drop of 0.224%. The source code of SparseByteNN will be available at https://github.com/lswzjuer/SparseByteNN
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Xu, H., Liu, S., Xu, Y., Wang, S., Li, J., Yan, C., Liangqiang, L., Fu, L., Pan, X., & Chen, F. (2023). SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2310.19509
MLA 9
Xu, Haitao, et al. “SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity.” arXiv (Cornell University), 2023. https://doi.org/10.48550/arxiv.2310.19509.
Chicago
Xu, Haitao, Songwei Liu, Yuyang Xu, Shuai Wang, Jiashi Li, Chenqian Yan, Li, Liangqiang, Lean Fu, Xin Pan, and Fangmin Chen. 2023. “SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2310.19509.
Harvard
Xu, H. et al. (2023) ‘SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity’, arXiv (Cornell University). doi:10.48550/arxiv.2310.19509.
IEEE
H. Xu et al., “SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity,” arXiv (Cornell University), 2023, doi: 10.48550/arxiv.2310.19509.
BibTeX
@misc{xu2023sparsebytenn,
title = {{SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity}},
author = {Xu, Haitao and Liu, Songwei and Xu, Yuyang and Wang, Shuai and Li, Jiashi and Yan, Chenqian and Liangqiang, Li, and Fu, Lean and Pan, Xin and Chen, Fangmin},
howpublished = {arXiv (Cornell University)},
year = {2023},
doi = {10.48550/arxiv.2310.19509},
url = {https://doi.org/10.48550/arxiv.2310.19509}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- sparse neural networks
- structured pruning
- unstructured pruning
- sparse kernels
- arms
- WebAssembly
- network rearrangement
- MobileNetV1
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Xu, H., Liu, S., Xu, Y., Wang, S., Li, J., Yan, C., Liangqiang, L., Fu, L., Pan, X., & Chen, F. (2023). SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2310.19509
How many times has this paper been cited?
5 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Haitao Xu, Songwei Liu, Yuyang Xu, Shuai Wang, Jiashi Li, Chenqian Yan, Li, Liangqiang, Lean Fu, Xin Pan, Fangmin Chen.
What is the DOI of this paper?
10.48550/arxiv.2310.19509, permanently at https://doi.org/10.48550/arxiv.2310.19509. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.