Preprint · 2026 · Open access

Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models

Zeyuan Chen, Yihan Ma, Xinyue Shen, Michael Backes, Yang Zhang

CISPA Helmholtz Center · Published 7 May 2026 · DOI 10.60882/cispa.32789088

Open full textCite this paper

In brief

“Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models” is a 2026 preprint by Zeyuan Chen and 4 co-authors, published in CISPA Helmholtz Center. OpenAlex counts 0 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at CISPA Helmholtz Center.

Citations
0
Authors
5

Abstract

Large language models (LLMs) show strong performance across many applications, but their ability to memorize and potentially reveal training data raises serious privacy concerns. We introduce the PopQuiz Attack, a black-box membership inference attack that tests whether a model can recall specific training examples. The core idea is to turn target data into quiz-style multiple-choice questions and infer membership from the model's answers. Across six widely used LLMs (GPT-3.5, GPT-4o, LLaMA2-7b, LLaMA2-13b, Mistral-7b, and Vicuna-7b) and four datasets, our method achieves an average ROC-AUC of 0.873 and outperforms existing approaches by 20.6%. We further analyze factors affecting attack success, including query complexity, data type, data structure, and training settings. We also evaluate instruction-based, filter-based, and differential privacy-based defenses, which reduce performance but do not eliminate the risk. Our results highlight persistent privacy vulnerabilities in modern LLMs.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Chen, Z., Ma, Y., Shen, X., Backes, M., & Zhang, Y. (2026). Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models. CISPA Helmholtz Center. https://doi.org/10.60882/cispa.32789088

MLA 9

Chen, Zeyuan, et al. “Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models.” CISPA Helmholtz Center, 2026. https://doi.org/10.60882/cispa.32789088.

Chicago

Chen, Zeyuan, Yihan Ma, Xinyue Shen, Michael Backes, and Yang Zhang. 2026. “Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models.” CISPA Helmholtz Center. https://doi.org/10.60882/cispa.32789088.

Harvard

Chen, Z. et al. (2026) ‘Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models’, CISPA Helmholtz Center. doi:10.60882/cispa.32789088.

IEEE

Z. Chen, Y. Ma, X. Shen, M. Backes, and Y. Zhang, “Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models,” CISPA Helmholtz Center, 2026, doi: 10.60882/cispa.32789088.

BibTeX

@misc{chen2026quiz,
  title     = {{Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models}},
  author    = {Chen, Zeyuan and Ma, Yihan and Shen, Xinyue and Backes, Michael and Zhang, Yang},
  howpublished = {CISPA Helmholtz Center},
  year      = {2026},
  doi       = {10.60882/cispa.32789088},
  url       = {https://doi.org/10.60882/cispa.32789088}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • membership inference attacks
  • large language models
  • black-box attacks
  • training data memorization
  • privacy vulnerabilities
  • data leakage

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Chen, Z., Ma, Y., Shen, X., Backes, M., & Zhang, Y. (2026). Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models. CISPA Helmholtz Center. https://doi.org/10.60882/cispa.32789088

How many times has this paper been cited?

0 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at CISPA Helmholtz Center: sign in and open the full text from this page.

Who wrote this paper?

Zeyuan Chen, Yihan Ma, Xinyue Shen, Michael Backes, Yang Zhang.

What is the DOI of this paper?

10.60882/cispa.32789088, permanently at https://doi.org/10.60882/cispa.32789088. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.