Preprint · 2025 · Open access
PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks
arXiv (Cornell University) · Published 16 January 2025 · DOI 10.48550/arxiv.2501.09367
In brief
“PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks” is a 2025 preprint by Huiyou Zhan and 6 co-authors, published in arXiv (Cornell University). OpenAlex counts 11 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 11
- Most cited in
- 2026 (7)
- References
- 38
- Authors
- 7
Abstract
Large language models (LLMs), while driving a new wave of interactive AI applications across numerous domains, suffer from high inference costs and heavy cloud dependency. Motivated by the redundancy phenomenon in linguistics, we propose a progressive inference paradigm over cloud and edge, i.e., firstly generating the sketch of the answer by LLMs at cloud, and then conducting parallel extension to fill in details by small models (SLMs) at edge. Progressive inference offers potential benefits to improve throughput and reduce inference latency while facing key implementation challenges, including decreased response quality from SLMs, a tradeoff between the brevity and comprehensiveness of sketches, as well as increased latency caused by network transmission and edge inference. In this work, we propose and implement PICE, an LLM serving system with semantic-level cloud-edge collaboration, enhancing inference throughput and quality through dynamic inference task scheduling, ensemble learning, and parallel edge inference. Extensive testbed experiments illustrate that our approach achieves $1.5-2\times$ throughput enhancement and up to 43% latency reduction, while also potentially enhancing the quality compared to SOTA systems.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Zhan, H., Xuan, Z., Tan, H., Han, T., Yong, D., Zhang, J., & Li, X. (2025). PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2501.09367
MLA 9
Zhan, Huiyou, et al. “PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2501.09367.
Chicago
Zhan, Huiyou, Zhang, Xuan, Haisheng Tan, Tian Han, Dongping Yong, Junyang Zhang, and Xiangyang Li. 2025. “PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2501.09367.
Harvard
Zhan, H. et al. (2025) ‘PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks’, arXiv (Cornell University). doi:10.48550/arxiv.2501.09367.
IEEE
H. Zhan et al., “PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2501.09367.
BibTeX
@misc{zhan2025pice,
title = {{PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks}},
author = {Zhan, Huiyou and Xuan, Zhang, and Tan, Haisheng and Han, Tian and Yong, Dongping and Zhang, Junyang and Li, Xiangyang},
howpublished = {arXiv (Cornell University)},
year = {2025},
doi = {10.48550/arxiv.2501.09367},
url = {https://doi.org/10.48550/arxiv.2501.09367}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- large language model serving
- cloud-edge collaboration
- ensemble learning
- small language models
- inference throughput
- inference latency reduction
- dynamic task scheduling
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Zhan, H., Xuan, Z., Tan, H., Han, T., Yong, D., Zhang, J., & Li, X. (2025). PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2501.09367
How many times has this paper been cited?
11 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Huiyou Zhan, Zhang, Xuan, Haisheng Tan, Tian Han, Dongping Yong, Junyang Zhang, Xiangyang Li.
What is the DOI of this paper?
10.48550/arxiv.2501.09367, permanently at https://doi.org/10.48550/arxiv.2501.09367. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.