Preprint · 2026 · Open access
Data Quality and Agentic Incidence in Online Samples
Published 9 January 2026 · DOI 10.17605/osf.io/m2asr
In brief
“Data Quality and Agentic Incidence in Online Samples” is a 2026 preprint by Andrew Gordon and 6 co-authors. OpenAlex counts 0 citations as of 18 November 2026. It is open access under a public domain licence: the full text is free to read.
- Citations
- 0
- Authors
- 7
Abstract
This study investigates two interrelated challenges facing online survey research: the potential infiltration of AI agents (automated systems completing surveys in place of human participants) and systematic differences in data quality across recruitment platforms. Purpose As large language models become increasingly capable, concerns have grown that AI agents may be completing online surveys (see Westwood, 2025), potentially compromising data integrity across behavioural and market research. Simultaneously, researchers lack updated comparative data on platform quality following the proliferation of LLM tools since 2022. This study addresses both issues through a large-scale, cross-platform comparison. Research Questions Research Question 1 examines AI agent incidence and impact across platforms. We will assess whether agent infiltration rates differ across platform types (direct/first-party panels, hybrid/managed networks, and marketplace/sample exchanges), characterise how agent responses differ from verified human responses, and quantify how agent presence distorts platform-level quality estimates. Research Question 2 compares data quality across ten major online recruitment platforms, updating and extending Peer et al. (2022). We examine whether platform type predicts data quality, whether pre-filtered samples outperform unfiltered samples, whether cost correlates with quality, and whether mobile respondents show lower quality than desktop users. Methods We will recruit 5,200 participants (n=400 per platform/condition) from ten platforms spanning three categories: direct panels (Prolific, MTurk, CloudResearch Connect, Verasight), hybrid networks (Prodege, Dynata), and marketplace exchanges (Prime Panels, Cint, Qualtrics Panels, Pure Spectrum). AI agents will be detected via automated environment checks and secondary behavioural indicators. Data quality will be assessed through behavioural measures (attention, honesty, consistency, comprehension, engagement, writing quality) and metadata indicators (fraud risk, ecosystem activity, completion metrics). Expected Outcomes We anticipate that marketplace/sample exchange platforms will show higher AI agent incidence than direct panels, that agents will exhibit distinctive response profiles exploitable for detection, and that data quality will vary substantially across platforms in ways not fully explained by cost. Findings will inform platform selection decisions and quality assurance practices for researchers using online samples.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Gordon, A., Rothschild, D. M., Jones, S., Sulik, J., Pepin, K., Hauser, D., & Affonso, F. M. (2026). Data Quality and Agentic Incidence in Online Samples. https://doi.org/10.17605/osf.io/m2asr
MLA 9
Gordon, Andrew, et al. “Data Quality and Agentic Incidence in Online Samples.” 2026. https://doi.org/10.17605/osf.io/m2asr.
Chicago
Gordon, Andrew, David M. Rothschild, Simon Jones, Justin Sulik, Karine Pepin, David Hauser, and Felipe M. Affonso. 2026. “Data Quality and Agentic Incidence in Online Samples.” https://doi.org/10.17605/osf.io/m2asr.
Harvard
Gordon, A. et al. (2026) ‘Data Quality and Agentic Incidence in Online Samples’. doi:10.17605/osf.io/m2asr.
IEEE
A. Gordon et al., “Data Quality and Agentic Incidence in Online Samples,” 2026, doi: 10.17605/osf.io/m2asr.
BibTeX
@misc{gordon2026data,
title = {{Data Quality and Agentic Incidence in Online Samples}},
author = {Gordon, Andrew and Rothschild, David M. and Jones, Simon and Sulik, Justin and Pepin, Karine and Hauser, David and Affonso, Felipe M.},
year = {2026},
doi = {10.17605/osf.io/m2asr},
url = {https://doi.org/10.17605/osf.io/m2asr}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Social Sciences › Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- recruitment websites
- large language models
- hybrid networks
- agent detection
- fraud risk
Work with this paper
Most-cited papers in Social Sciences
- Using thematic analysis in psychologyVirginia Braun, Victoria Clarke · Qualitative Research in Psychology · 2006 · 240,222 citations
- Case Study Research: Design and MethodsRobert K. Yin · 1984 · 81,951 citations
- Three Approaches to Qualitative Content AnalysisHsiu-Fang Hsieh, Sarah Elizabeth Shannon · Qualitative Health Research · 2005 · 49,623 citations
- Qualitative Data Analysis: An Expanded SourcebookMatthew B. Miles, A Michael Huberman · Journal of Environmental Psychology · 1994 · 48,311 citations
- Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groupsAllison Tong, Peter Sainsbury, Jonathan C. Craig · International Journal for Quality in Health Care · 2007 · 45,353 citations
Questions about this paper
How do I cite this paper in APA style?
Gordon, A., Rothschild, D. M., Jones, S., Sulik, J., Pepin, K., Hauser, D., & Affonso, F. M. (2026). Data Quality and Agentic Incidence in Online Samples. https://doi.org/10.17605/osf.io/m2asr
How many times has this paper been cited?
0 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: sign in and open the full text from this page.
Who wrote this paper?
Andrew Gordon, David M. Rothschild, Simon Jones, Justin Sulik, Karine Pepin, David Hauser, Felipe M. Affonso.
What is the DOI of this paper?
10.17605/osf.io/m2asr, permanently at https://doi.org/10.17605/osf.io/m2asr. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.