Preprint · 2025 · Open access
PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty
arXiv (Cornell University) · Published 24 June 2025 · DOI 10.48550/arxiv.2506.19563
In brief
“PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty” is a 2025 preprint by Jinwen He and 4 co-authors, published in arXiv (Cornell University). OpenAlex counts 1 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 1
- Most cited in
- 2026 (1)
- References
- 34
- Authors
- 5
Abstract
Large Language Models (LLMs) are widely used in sensitive domains, including healthcare, finance, and legal services, raising concerns about potential private information leaks during inference. Privacy extraction attacks, such as jailbreaking, expose vulnerabilities in LLMs by crafting inputs that force the models to output sensitive information. However, these attacks cannot verify whether the extracted private information is accurate, as no public datasets exist for cross-validation, leaving a critical gap in private information detection during inference. To address this, we propose PrivacyXray, a novel framework detecting privacy breaches by analyzing LLM inner states. Our analysis reveals that LLMs exhibit higher semantic coherence and probabilistic certainty when generating correct private outputs. Based on this, PrivacyXray detects privacy breaches using four metrics: intra-layer and inter-layer semantic similarity, token-level and sentence-level probability distributions. PrivacyXray addresses critical challenges in private information detection by overcoming the lack of open-source private datasets and eliminating reliance on external data for validation. It achieves this through the synthesis of realistic private data and a detection mechanism based on the inner states of LLMs. Experiments show that PrivacyXray achieves consistent performance, with an average accuracy of 92.69% across five LLMs. Compared to state-of-the-art methods, PrivacyXray achieves significant improvements, with an average accuracy increase of 20.06%, highlighting its stability and practical utility in real-world applications.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
He, J., Lu, Y., Lin, Z., Kai, C., & Zhao, Y. (2025). PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2506.19563
MLA 9
He, Jinwen, et al. “PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2506.19563.
Chicago
He, Jinwen, Yi Lu, Zijin Lin, Chen, Kai, and Yue Zhao. 2025. “PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2506.19563.
Harvard
He, J. et al. (2025) ‘PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty’, arXiv (Cornell University). doi:10.48550/arxiv.2506.19563.
IEEE
J. He, Y. Lu, Z. Lin, C. Kai, and Y. Zhao, “PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2506.19563.
BibTeX
@misc{he2025privacyxray,
title = {{PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty}},
author = {He, Jinwen and Lu, Yi and Lin, Zijin and Kai, Chen, and Zhao, Yue},
howpublished = {arXiv (Cornell University)},
year = {2025},
doi = {10.48550/arxiv.2506.19563},
url = {https://doi.org/10.48550/arxiv.2506.19563}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- large language models
- jailbreaking
- semantic consistency
- semantic similarity
- inference privacy
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
He, J., Lu, Y., Lin, Z., Kai, C., & Zhao, Y. (2025). PrivacyXray: Detecting Privacy Breaches in LLMs through Semantic Consistency and Probability Certainty. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2506.19563
How many times has this paper been cited?
1 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Jinwen He, Yi Lu, Zijin Lin, Chen, Kai, Yue Zhao.
What is the DOI of this paper?
10.48550/arxiv.2506.19563, permanently at https://doi.org/10.48550/arxiv.2506.19563. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.