Conference paper · 2024 · Open access
LLM-PBE: Assessing Data Privacy in Large Language Models
Proceedings of the VLDB Endowment, 17(11), 3201–3214 · Published 1 July 2024 · DOI 10.14778/3681954.3681994
In brief
“LLM-PBE: Assessing Data Privacy in Large Language Models” is a 2024 conference paper by Qinbin Li and 12 co-authors, published in Proceedings of the VLDB Endowment. OpenAlex counts 153 citations as of 19 November 2026, which puts it in the top 2% of works published in 2024 by citations. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 153
- Most cited in
- 2025 (75)
- Field-weighted impact
- 12.62
- References
- 115
- Authors
- 13
Abstract
Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis. Their profound capabilities in processing and interpreting complex language data, however, bring to light pressing concerns regarding data privacy, especially the risk of unintentional training data leakage. Despite the critical nature of this issue, there has been no existing literature to offer a comprehensive assessment of data privacy risks in LLMs. Addressing this gap, our paper introduces LLM-PBE, a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs. LLM-PBE is designed to analyze privacy across the entire lifecycle of LLMs, incorporating diverse attack and defense strategies, and handling various data types and metrics. Through detailed experimentation with multiple LLMs, LLM-PBE facilitates an in-depth exploration of data privacy concerns, shedding light on influential factors such as model size, data characteristics, and evolving temporal dimensions. This study not only enriches the understanding of privacy issues in LLMs but also serves as a vital resource for future research in the field. Aimed at enhancing the breadth of knowledge in this area, the findings, resources, and our full technical report are made available at https://llm-pbe.github.io/, providing an open platform for academic and practical advancements in LLM privacy assessment.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Li, Q., Hong, J., Xie, C., Tan, J. T. C., Xin, R., Hou, J., Yin, X., Wang, Z., Hendrycks, D., Wang, Z., Li, B., He, B., & Song, D. X. (2024). LLM-PBE: Assessing Data Privacy in Large Language Models. Proceedings of the VLDB Endowment, 17(11), 3201–3214. https://doi.org/10.14778/3681954.3681994
MLA 9
Li, Qinbin, et al. “LLM-PBE: Assessing Data Privacy in Large Language Models.” Proceedings of the VLDB Endowment, vol. 17, no. 11, 2024, pp. 3201–3214. https://doi.org/10.14778/3681954.3681994.
Chicago
Li, Qinbin, Junyuan Hong, Chulin Xie, Jeffrey Too Chuan Tan, Rachel Xin, Junyi Hou, Xavier Yin, et al. 2024. “LLM-PBE: Assessing Data Privacy in Large Language Models.” Proceedings of the VLDB Endowment 17 (11): 3201–3214. https://doi.org/10.14778/3681954.3681994.
Harvard
Li, Q. et al. (2024) ‘LLM-PBE: Assessing Data Privacy in Large Language Models’, Proceedings of the VLDB Endowment, 17(11), pp. 3201–3214. doi:10.14778/3681954.3681994.
IEEE
Q. Li et al., “LLM-PBE: Assessing Data Privacy in Large Language Models,” Proceedings of the VLDB Endowment, vol. 17, no. 11, pp. 3201–3214, 2024, doi: 10.14778/3681954.3681994.
BibTeX
@inproceedings{li2024assessing,
title = {{LLM-PBE: Assessing Data Privacy in Large Language Models}},
author = {Li, Qinbin and Hong, Junyuan and Xie, Chulin and Tan, Jeffrey Too Chuan and Xin, Rachel and Hou, Junyi and Yin, Xavier and Wang, Zhun and Hendrycks, Dan and Wang, Zhangyang and Li, Bo and He, Bingsheng and Song, Dawn Xiaodong},
booktitle = {Proceedings of the VLDB Endowment},
year = {2024},
volume = {17},
number = {11},
pages = {3201--3214},
doi = {10.14778/3681954.3681994},
url = {https://doi.org/10.14778/3681954.3681994}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2024 | 9 | |
| 2025 | 75 | |
| 2026 | 69 |
Its field-weighted citation impact is 12.62: it has been cited 12.62 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- large language models
- data privacy
- training data leakage
- privacy risk assessment
- model size
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Li, Q., Hong, J., Xie, C., Tan, J. T. C., Xin, R., Hou, J., Yin, X., Wang, Z., Hendrycks, D., Wang, Z., Li, B., He, B., & Song, D. X. (2024). LLM-PBE: Assessing Data Privacy in Large Language Models. Proceedings of the VLDB Endowment, 17(11), 3201–3214. https://doi.org/10.14778/3681954.3681994
How many times has this paper been cited?
153 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Qinbin Li, Junyuan Hong, Chulin Xie, Jeffrey Too Chuan Tan, Rachel Xin, Junyi Hou, Xavier Yin, Zhun Wang, Dan Hendrycks, Zhangyang Wang, Bo Li, Bingsheng He and 1 more.
What is the DOI of this paper?
10.14778/3681954.3681994, permanently at https://doi.org/10.14778/3681954.3681994. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.