Preprint · 2026 · Open access
Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models
CISPA Helmholtz Center · Published 7 May 2026 · DOI 10.60882/cispa.32789088
In brief
“Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models” is a 2026 preprint by Zeyuan Chen and 4 co-authors, published in CISPA Helmholtz Center. OpenAlex counts 0 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at CISPA Helmholtz Center.
- Citations
- 0
- Authors
- 5
Abstract
Large language models (LLMs) show strong performance across many applications, but their ability to memorize and potentially reveal training data raises serious privacy concerns. We introduce the PopQuiz Attack, a black-box membership inference attack that tests whether a model can recall specific training examples. The core idea is to turn target data into quiz-style multiple-choice questions and infer membership from the model's answers. Across six widely used LLMs (GPT-3.5, GPT-4o, LLaMA2-7b, LLaMA2-13b, Mistral-7b, and Vicuna-7b) and four datasets, our method achieves an average ROC-AUC of 0.873 and outperforms existing approaches by 20.6%. We further analyze factors affecting attack success, including query complexity, data type, data structure, and training settings. We also evaluate instruction-based, filter-based, and differential privacy-based defenses, which reduce performance but do not eliminate the risk. Our results highlight persistent privacy vulnerabilities in modern LLMs.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Chen, Z., Ma, Y., Shen, X., Backes, M., & Zhang, Y. (2026). Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models. CISPA Helmholtz Center. https://doi.org/10.60882/cispa.32789088
MLA 9
Chen, Zeyuan, et al. “Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models.” CISPA Helmholtz Center, 2026. https://doi.org/10.60882/cispa.32789088.
Chicago
Chen, Zeyuan, Yihan Ma, Xinyue Shen, Michael Backes, and Yang Zhang. 2026. “Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models.” CISPA Helmholtz Center. https://doi.org/10.60882/cispa.32789088.
Harvard
Chen, Z. et al. (2026) ‘Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models’, CISPA Helmholtz Center. doi:10.60882/cispa.32789088.
IEEE
Z. Chen, Y. Ma, X. Shen, M. Backes, and Y. Zhang, “Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models,” CISPA Helmholtz Center, 2026, doi: 10.60882/cispa.32789088.
BibTeX
@misc{chen2026quiz,
title = {{Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models}},
author = {Chen, Zeyuan and Ma, Yihan and Shen, Xinyue and Backes, Michael and Zhang, Yang},
howpublished = {CISPA Helmholtz Center},
year = {2026},
doi = {10.60882/cispa.32789088},
url = {https://doi.org/10.60882/cispa.32789088}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- membership inference attacks
- large language models
- black-box attacks
- training data memorization
- privacy vulnerabilities
- data leakage
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Chen, Z., Ma, Y., Shen, X., Backes, M., & Zhang, Y. (2026). Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models. CISPA Helmholtz Center. https://doi.org/10.60882/cispa.32789088
How many times has this paper been cited?
0 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at CISPA Helmholtz Center: sign in and open the full text from this page.
Who wrote this paper?
Zeyuan Chen, Yihan Ma, Xinyue Shen, Michael Backes, Yang Zhang.
What is the DOI of this paper?
10.60882/cispa.32789088, permanently at https://doi.org/10.60882/cispa.32789088. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.