Preprint · 2025 · Open access

PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks

Huiyou Zhan, Zhang, Xuan, Haisheng Tan, Tian Han, Dongping Yong, Junyang Zhang, Xiangyang Li

arXiv (Cornell University) · Published 16 January 2025 · DOI 10.48550/arxiv.2501.09367

Open PDFView at the publisherCite this paper

In brief

“PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks” is a 2025 preprint by Huiyou Zhan and 6 co-authors, published in arXiv (Cornell University). OpenAlex counts 11 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
11
Most cited in
2026 (7)
References
38
Authors
7

Abstract

Large language models (LLMs), while driving a new wave of interactive AI applications across numerous domains, suffer from high inference costs and heavy cloud dependency. Motivated by the redundancy phenomenon in linguistics, we propose a progressive inference paradigm over cloud and edge, i.e., firstly generating the sketch of the answer by LLMs at cloud, and then conducting parallel extension to fill in details by small models (SLMs) at edge. Progressive inference offers potential benefits to improve throughput and reduce inference latency while facing key implementation challenges, including decreased response quality from SLMs, a tradeoff between the brevity and comprehensiveness of sketches, as well as increased latency caused by network transmission and edge inference. In this work, we propose and implement PICE, an LLM serving system with semantic-level cloud-edge collaboration, enhancing inference throughput and quality through dynamic inference task scheduling, ensemble learning, and parallel edge inference. Extensive testbed experiments illustrate that our approach achieves $1.5-2\times$ throughput enhancement and up to 43% latency reduction, while also potentially enhancing the quality compared to SOTA systems.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Zhan, H., Xuan, Z., Tan, H., Han, T., Yong, D., Zhang, J., & Li, X. (2025). PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2501.09367

MLA 9

Zhan, Huiyou, et al. “PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2501.09367.

Chicago

Zhan, Huiyou, Zhang, Xuan, Haisheng Tan, Tian Han, Dongping Yong, Junyang Zhang, and Xiangyang Li. 2025. “PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2501.09367.

Harvard

Zhan, H. et al. (2025) ‘PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks’, arXiv (Cornell University). doi:10.48550/arxiv.2501.09367.

IEEE

H. Zhan et al., “PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2501.09367.

BibTeX

@misc{zhan2025pice,
  title     = {{PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks}},
  author    = {Zhan, Huiyou and Xuan, Zhang, and Tan, Haisheng and Han, Tian and Yong, Dongping and Zhang, Junyang and Li, Xiangyang},
  howpublished = {arXiv (Cornell University)},
  year      = {2025},
  doi       = {10.48550/arxiv.2501.09367},
  url       = {https://doi.org/10.48550/arxiv.2501.09367}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing

  • large language model serving
  • cloud-edge collaboration
  • ensemble learning
  • small language models
  • inference throughput
  • inference latency reduction
  • dynamic task scheduling

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Zhan, H., Xuan, Z., Tan, H., Han, T., Yong, D., Zhang, J., & Li, X. (2025). PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networks. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2501.09367

How many times has this paper been cited?

11 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Huiyou Zhan, Zhang, Xuan, Haisheng Tan, Tian Han, Dongping Yong, Junyang Zhang, Xiangyang Li.

What is the DOI of this paper?

10.48550/arxiv.2501.09367, permanently at https://doi.org/10.48550/arxiv.2501.09367. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.