Journal article · 2021 · Open access

GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity

Wei Niu, Zhengang Li, Xiaolong Ma, Peiyan Dong, Gang Zhou, Xuehai Qian, Xue Lin, Yanzhi Wang, Bin Ren

IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 6224–6239 · Published 16 June 2021 · DOI 10.1109/tpami.2021.3089687

Open PDFView at the publisherCite this paper

In brief

“GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity” is a 2021 journal article by Wei Niu and 8 co-authors, published in IEEE Transactions on Pattern Analysis and Machine Intelligence. OpenAlex counts 31 citations as of 19 November 2026, which puts it in the top 6% of works published in 2021 by citations. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
31
Most cited in
2022 (10)
Field-weighted impact
1.61
References
69
Authors
9

Abstract

It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices, because even the powerful modern mobile devices are considered as "resource-constrained" when executing large-scale DNNs. It necessitates the sparse model inference via weight pruning, i.e., DNN weight sparsity, and it is desirable to design a new DNN weight sparsity scheme that can facilitate real-time inference on mobile devices while preserving a high sparse model accuracy. This paper designs a novel mobile inference acceleration framework GRIM that is General to both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) and that achieves Real-time execution and high accuracy, leveraging fine-grained structured sparse model Inference and compiler optimizations for Mobiles. We start by proposing a new fine-grained structured sparsity scheme through the Block-based Column-Row (BCR) pruning. Based on this new fine-grained structured sparsity, our GRIM framework consists of two parts: (a) the compiler optimization and code generation for real-time mobile inference; and (b) the BCR pruning optimizations for determining pruning hyperparameters and performing weight pruning. We compare GRIM with Alibaba MNN, TVM, TensorFlow-Lite, a sparse implementation based on CSR, PatDNN, and ESE (a representative FPGA inference acceleration framework for RNNs), and achieve up to 14.08× speedup.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Niu, W., Li, Z., Ma, X., Dong, P., Zhou, G., Qian, X., Lin, X., Wang, Y., & Ren, B. (2021). GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 6224–6239. https://doi.org/10.1109/tpami.2021.3089687

MLA 9

Niu, Wei, et al. “GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity.” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, 2021, pp. 6224–6239. https://doi.org/10.1109/tpami.2021.3089687.

Chicago

Niu, Wei, Zhengang Li, Xiaolong Ma, Peiyan Dong, Gang Zhou, Xuehai Qian, Xue Lin, Yanzhi Wang, and Bin Ren. 2021. “GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity.” IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (10): 6224–6239. https://doi.org/10.1109/tpami.2021.3089687.

Harvard

Niu, W. et al. (2021) ‘GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity’, IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), pp. 6224–6239. doi:10.1109/tpami.2021.3089687.

IEEE

W. Niu et al., “GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6224–6239, 2021, doi: 10.1109/tpami.2021.3089687.

BibTeX

@article{niu2021grim,
  title     = {{GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity}},
  author    = {Niu, Wei and Li, Zhengang and Ma, Xiaolong and Dong, Peiyan and Zhou, Gang and Qian, Xuehai and Lin, Xue and Wang, Yanzhi and Ren, Bin},
  journal   = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year      = {2021},
  volume    = {44},
  number    = {10},
  pages     = {6224--6239},
  doi       = {10.1109/tpami.2021.3089687},
  url       = {https://doi.org/10.1109/tpami.2021.3089687}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202155
20221010
202355
202433
202555
202633

Its field-weighted citation impact is 1.61: it has been cited 1.61 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications

  • deep neural network inference
  • mobile devices
  • weight sparsity
  • weight pruning
  • structured sparsity
  • real-time inference
  • compiler optimization
  • code generation

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Niu, W., Li, Z., Ma, X., Dong, P., Zhou, G., Qian, X., Lin, X., Wang, Y., & Ren, B. (2021). GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices Based on Fine-Grained Structured Weight Sparsity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 6224–6239. https://doi.org/10.1109/tpami.2021.3089687

How many times has this paper been cited?

31 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Wei Niu, Zhengang Li, Xiaolong Ma, Peiyan Dong, Gang Zhou, Xuehai Qian, Xue Lin, Yanzhi Wang, Bin Ren.

What is the DOI of this paper?

10.1109/tpami.2021.3089687, permanently at https://doi.org/10.1109/tpami.2021.3089687. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.