Preprint · 2026 · Open access

A scale for detecting LLM-generated responses in online survey research

Cameron S. Kay, Madalina Vlasceanu

PsyArXiv (OSF Preprints) · Published 21 March 2026

Open full textCite this paper

In brief

“A scale for detecting LLM-generated responses in online survey research” is a 2026 preprint by Cameron S. Kay and Madalina Vlasceanu, published in PsyArXiv (OSF Preprints). OpenAlex counts 0 citations as of 18 November 2026. It is open access under a CC BY licence: the full text is free to read at PsyArXiv (OSF Preprints).

Citations
0
Authors
2

Abstract

The rise of LLM-generated respondents poses a growing threat to the integrity of online survey research, yet available detection techniques remain opaque, proprietary, or poorly matched to the problem. In the present project, we propose and evaluate ECLAIR (Exploiting Common Limitations of AI Respondents), a novel framework for developing AI-detection items. We generated synthetic respondents using six LLMs (NTotal = 1,800) and compared them to census-matched samples of human participants recruited from five online data-collection platforms (NTotal = 1,518). Using a set of common tasks and scales drawn from the social sciences (e.g., the Trolley Problem; the Cognitive Reflection Test), we found that response distributions produced by synthetic respondents and human respondents diverge substantially, such that a researcher who relied solely on the former would be liable to misestimate experimental effects and systematically underestimate individual differences. Critically, we also found that a set of 22 items designed using the ECLAIR framework could successfully discriminate between the two types of respondents (AUC = .990), correctly classifying 98.78% of synthetic respondents and 95.78% of human respondents. A cross-validated short-form of four items preserved much of this accuracy (AUC = .980), correctly classifying 97.78% of synthetic respondents and 96.05% of human respondents. Although this project demonstrates that synthetic respondents pose a serious threat to online survey research, it also shows that novel tools, such as the ECLAIR framework, can help mitigate that threat.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Kay, C. S., & Vlasceanu, M. (2026). A scale for detecting LLM-generated responses in online survey research. PsyArXiv (OSF Preprints). https://osf.io/4p7ns

MLA 9

Kay, Cameron S., and Madalina Vlasceanu. “A scale for detecting LLM-generated responses in online survey research.” PsyArXiv (OSF Preprints), 2026. https://osf.io/4p7ns.

Chicago

Kay, Cameron S. and Madalina Vlasceanu. 2026. “A scale for detecting LLM-generated responses in online survey research.” PsyArXiv (OSF Preprints). https://osf.io/4p7ns.

Harvard

Kay, C. S. and Vlasceanu, M. (2026) ‘A scale for detecting LLM-generated responses in online survey research’, PsyArXiv (OSF Preprints). Available at: https://osf.io/4p7ns.

IEEE

C. S. Kay and M. Vlasceanu, “A scale for detecting LLM-generated responses in online survey research,” PsyArXiv (OSF Preprints), 2026.

BibTeX

@misc{kay2026scale,
  title     = {{A scale for detecting LLM-generated responses in online survey research}},
  author    = {Kay, Cameron S. and Vlasceanu, Madalina},
  howpublished = {PsyArXiv (OSF Preprints)},
  year      = {2026},
  url       = {https://osf.io/4p7ns}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Social Sciences › Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse

  • AI-generated responses
  • large language models
  • AI detection
  • response distribution
  • cognitive reflection
  • trolley problem
  • survey data quality

Work with this paper

Most-cited papers in Social Sciences

All highly cited Social Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Kay, C. S., & Vlasceanu, M. (2026). A scale for detecting LLM-generated responses in online survey research. PsyArXiv (OSF Preprints). https://osf.io/4p7ns

How many times has this paper been cited?

0 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at PsyArXiv (OSF Preprints): sign in and open the full text from this page.

Who wrote this paper?

Cameron S. Kay, Madalina Vlasceanu.

Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.