Preprint · 2026 · Open access
A scale for detecting LLM-generated responses in online survey research
PsyArXiv (OSF Preprints) · Published 21 March 2026
In brief
“A scale for detecting LLM-generated responses in online survey research” is a 2026 preprint by Cameron S. Kay and Madalina Vlasceanu, published in PsyArXiv (OSF Preprints). OpenAlex counts 0 citations as of 18 November 2026. It is open access under a CC BY licence: the full text is free to read at PsyArXiv (OSF Preprints).
- Citations
- 0
- Authors
- 2
Abstract
The rise of LLM-generated respondents poses a growing threat to the integrity of online survey research, yet available detection techniques remain opaque, proprietary, or poorly matched to the problem. In the present project, we propose and evaluate ECLAIR (Exploiting Common Limitations of AI Respondents), a novel framework for developing AI-detection items. We generated synthetic respondents using six LLMs (NTotal = 1,800) and compared them to census-matched samples of human participants recruited from five online data-collection platforms (NTotal = 1,518). Using a set of common tasks and scales drawn from the social sciences (e.g., the Trolley Problem; the Cognitive Reflection Test), we found that response distributions produced by synthetic respondents and human respondents diverge substantially, such that a researcher who relied solely on the former would be liable to misestimate experimental effects and systematically underestimate individual differences. Critically, we also found that a set of 22 items designed using the ECLAIR framework could successfully discriminate between the two types of respondents (AUC = .990), correctly classifying 98.78% of synthetic respondents and 95.78% of human respondents. A cross-validated short-form of four items preserved much of this accuracy (AUC = .980), correctly classifying 97.78% of synthetic respondents and 96.05% of human respondents. Although this project demonstrates that synthetic respondents pose a serious threat to online survey research, it also shows that novel tools, such as the ECLAIR framework, can help mitigate that threat.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Kay, C. S., & Vlasceanu, M. (2026). A scale for detecting LLM-generated responses in online survey research. PsyArXiv (OSF Preprints). https://osf.io/4p7ns
MLA 9
Kay, Cameron S., and Madalina Vlasceanu. “A scale for detecting LLM-generated responses in online survey research.” PsyArXiv (OSF Preprints), 2026. https://osf.io/4p7ns.
Chicago
Kay, Cameron S. and Madalina Vlasceanu. 2026. “A scale for detecting LLM-generated responses in online survey research.” PsyArXiv (OSF Preprints). https://osf.io/4p7ns.
Harvard
Kay, C. S. and Vlasceanu, M. (2026) ‘A scale for detecting LLM-generated responses in online survey research’, PsyArXiv (OSF Preprints). Available at: https://osf.io/4p7ns.
IEEE
C. S. Kay and M. Vlasceanu, “A scale for detecting LLM-generated responses in online survey research,” PsyArXiv (OSF Preprints), 2026.
BibTeX
@misc{kay2026scale,
title = {{A scale for detecting LLM-generated responses in online survey research}},
author = {Kay, Cameron S. and Vlasceanu, Madalina},
howpublished = {PsyArXiv (OSF Preprints)},
year = {2026},
url = {https://osf.io/4p7ns}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Social Sciences › Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- AI-generated responses
- large language models
- AI detection
- response distribution
- cognitive reflection
- trolley problem
- survey data quality
Work with this paper
Most-cited papers in Social Sciences
- Using thematic analysis in psychologyVirginia Braun, Victoria Clarke · Qualitative Research in Psychology · 2006 · 240,222 citations
- Case Study Research: Design and MethodsRobert K. Yin · 1984 · 81,951 citations
- Three Approaches to Qualitative Content AnalysisHsiu-Fang Hsieh, Sarah Elizabeth Shannon · Qualitative Health Research · 2005 · 49,623 citations
- Qualitative Data Analysis: An Expanded SourcebookMatthew B. Miles, A Michael Huberman · Journal of Environmental Psychology · 1994 · 48,311 citations
- Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groupsAllison Tong, Peter Sainsbury, Jonathan C. Craig · International Journal for Quality in Health Care · 2007 · 45,353 citations
Questions about this paper
How do I cite this paper in APA style?
Kay, C. S., & Vlasceanu, M. (2026). A scale for detecting LLM-generated responses in online survey research. PsyArXiv (OSF Preprints). https://osf.io/4p7ns
How many times has this paper been cited?
0 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at PsyArXiv (OSF Preprints): sign in and open the full text from this page.
Who wrote this paper?
Cameron S. Kay, Madalina Vlasceanu.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.