Journal article · 2026 · Open access
AI-generated language remains systematically distinguishable from human communication
Proceedings of the National Academy of Sciences, 123(42), e2531748123 · Published 12 October 2026 · DOI 10.1073/pnas.2531748123
In brief
“AI-generated language remains systematically distinguishable from human communication” is a 2026 journal article by Nicolò Pagan and 3 co-authors, published in Proceedings of the National Academy of Sciences. OpenAlex counts 10 citations as of 17 November 2026, which puts it in the top 2% of works published in 2026 by citations. It is open access under a CC BY licence: the full text is free to read at Proceedings of the National Academy of Sciences.
- Citations
- 10
- Most cited in
- 2026 (10)
- Field-weighted impact
- 13.95
- References
- 51
- Authors
- 4
Abstract
Large language models (LLMs) are increasingly used in the social sciences to simulate human behavior and communication, yet validation methods for these applications remain limited. Existing evaluations often rely on human judgments of plausibility, a coarse, difficult-to-scale, and often unreliable benchmark for distinguishing AI-generated from human text. As a result, researchers lack robust tools for assessing the realism of LLM-generated language and for calibrating models to real-world data. We address this gap in two ways. First, we introduce the Conversational Human Authenticity Test (CHAT), a validation framework that combines aggregate metrics of detectability and semantic similarity with interpretable linguistic features, including stylistic markers and topical patterns, to assess how closely LLMs approximate human language in a given dataset. Second, we benchmark nine open-weight LLMs across five calibration strategies, including fine-tuning, stylistic prompting, and context retrieval, on user interactions from X, Bluesky, and Reddit. We find that even after calibration, LLM outputs remain clearly distinguishable from human text, especially in affective tone and emotional expression. Instruction-tuned models generally underperform base models, and larger models do not obviously improve human-likeness. We also identify a trade-off between realism and semantic fidelity: methods that make outputs seem more human often move them further from the content of human responses. Together, these results provide a scalable framework for validation and calibration in LLM-based social simulation while underscoring current limits in reproducing human communication.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Pagan, N., Törnberg, P., Bail, C. A., & Barrie, C. (2026). AI-generated language remains systematically distinguishable from human communication. Proceedings of the National Academy of Sciences, 123(42), e2531748123. https://doi.org/10.1073/pnas.2531748123
MLA 9
Pagan, Nicolò, et al. “AI-generated language remains systematically distinguishable from human communication.” Proceedings of the National Academy of Sciences, vol. 123, no. 42, 2026, p. e2531748123. https://doi.org/10.1073/pnas.2531748123.
Chicago
Pagan, Nicolò, Petter Törnberg, Christopher A. Bail, and Christopher Barrie. 2026. “AI-generated language remains systematically distinguishable from human communication.” Proceedings of the National Academy of Sciences 123 (42): e2531748123. https://doi.org/10.1073/pnas.2531748123.
Harvard
Pagan, N. et al. (2026) ‘AI-generated language remains systematically distinguishable from human communication’, Proceedings of the National Academy of Sciences, 123(42), p. e2531748123. doi:10.1073/pnas.2531748123.
IEEE
N. Pagan, P. Törnberg, C. A. Bail, and C. Barrie, “AI-generated language remains systematically distinguishable from human communication,” Proceedings of the National Academy of Sciences, vol. 123, no. 42, p. e2531748123, 2026, doi: 10.1073/pnas.2531748123.
BibTeX
@article{pagan2026generated,
title = {{AI-generated language remains systematically distinguishable from human communication}},
author = {Pagan, Nicolò and Törnberg, Petter and Bail, Christopher A. and Barrie, Christopher},
journal = {Proceedings of the National Academy of Sciences},
year = {2026},
volume = {123},
number = {42},
pages = {e2531748123},
doi = {10.1073/pnas.2531748123},
url = {https://doi.org/10.1073/pnas.2531748123}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- large language models
- AI-generated text detection
- human-likeness
- LLM calibration
- social simulation
- semantic similarity
- emotional tone
- emotional expression
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Pagan, N., Törnberg, P., Bail, C. A., & Barrie, C. (2026). AI-generated language remains systematically distinguishable from human communication. Proceedings of the National Academy of Sciences, 123(42), e2531748123. https://doi.org/10.1073/pnas.2531748123
How many times has this paper been cited?
10 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Proceedings of the National Academy of Sciences: sign in and open the full text from this page.
Who wrote this paper?
Nicolò Pagan, Petter Törnberg, Christopher A. Bail, Christopher Barrie.
What is the DOI of this paper?
10.1073/pnas.2531748123, permanently at https://doi.org/10.1073/pnas.2531748123. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.