Journal article · 2026 · Open access

AI-generated language remains systematically distinguishable from human communication

Nicolò Pagan, Petter Törnberg, Christopher A. Bail, Christopher Barrie

Proceedings of the National Academy of Sciences, 123(42), e2531748123 · Published 12 October 2026 · DOI 10.1073/pnas.2531748123

Open full textCite this paper

In brief

“AI-generated language remains systematically distinguishable from human communication” is a 2026 journal article by Nicolò Pagan and 3 co-authors, published in Proceedings of the National Academy of Sciences. OpenAlex counts 10 citations as of 17 November 2026, which puts it in the top 2% of works published in 2026 by citations. It is open access under a CC BY licence: the full text is free to read at Proceedings of the National Academy of Sciences.

Citations
10
Most cited in
2026 (10)
Field-weighted impact
13.95
References
51
Authors
4

Abstract

Large language models (LLMs) are increasingly used in the social sciences to simulate human behavior and communication, yet validation methods for these applications remain limited. Existing evaluations often rely on human judgments of plausibility, a coarse, difficult-to-scale, and often unreliable benchmark for distinguishing AI-generated from human text. As a result, researchers lack robust tools for assessing the realism of LLM-generated language and for calibrating models to real-world data. We address this gap in two ways. First, we introduce the Conversational Human Authenticity Test (CHAT), a validation framework that combines aggregate metrics of detectability and semantic similarity with interpretable linguistic features, including stylistic markers and topical patterns, to assess how closely LLMs approximate human language in a given dataset. Second, we benchmark nine open-weight LLMs across five calibration strategies, including fine-tuning, stylistic prompting, and context retrieval, on user interactions from X, Bluesky, and Reddit. We find that even after calibration, LLM outputs remain clearly distinguishable from human text, especially in affective tone and emotional expression. Instruction-tuned models generally underperform base models, and larger models do not obviously improve human-likeness. We also identify a trade-off between realism and semantic fidelity: methods that make outputs seem more human often move them further from the content of human responses. Together, these results provide a scalable framework for validation and calibration in LLM-based social simulation while underscoring current limits in reproducing human communication.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Pagan, N., Törnberg, P., Bail, C. A., & Barrie, C. (2026). AI-generated language remains systematically distinguishable from human communication. Proceedings of the National Academy of Sciences, 123(42), e2531748123. https://doi.org/10.1073/pnas.2531748123

MLA 9

Pagan, Nicolò, et al. “AI-generated language remains systematically distinguishable from human communication.” Proceedings of the National Academy of Sciences, vol. 123, no. 42, 2026, p. e2531748123. https://doi.org/10.1073/pnas.2531748123.

Chicago

Pagan, Nicolò, Petter Törnberg, Christopher A. Bail, and Christopher Barrie. 2026. “AI-generated language remains systematically distinguishable from human communication.” Proceedings of the National Academy of Sciences 123 (42): e2531748123. https://doi.org/10.1073/pnas.2531748123.

Harvard

Pagan, N. et al. (2026) ‘AI-generated language remains systematically distinguishable from human communication’, Proceedings of the National Academy of Sciences, 123(42), p. e2531748123. doi:10.1073/pnas.2531748123.

IEEE

N. Pagan, P. Törnberg, C. A. Bail, and C. Barrie, “AI-generated language remains systematically distinguishable from human communication,” Proceedings of the National Academy of Sciences, vol. 123, no. 42, p. e2531748123, 2026, doi: 10.1073/pnas.2531748123.

BibTeX

@article{pagan2026generated,
  title     = {{AI-generated language remains systematically distinguishable from human communication}},
  author    = {Pagan, Nicolò and Törnberg, Petter and Bail, Christopher A. and Barrie, Christopher},
  journal   = {Proceedings of the National Academy of Sciences},
  year      = {2026},
  volume    = {123},
  number    = {42},
  pages     = {e2531748123},
  doi       = {10.1073/pnas.2531748123},
  url       = {https://doi.org/10.1073/pnas.2531748123}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling

  • large language models
  • AI-generated text detection
  • human-likeness
  • LLM calibration
  • social simulation
  • semantic similarity
  • emotional tone
  • emotional expression

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Pagan, N., Törnberg, P., Bail, C. A., & Barrie, C. (2026). AI-generated language remains systematically distinguishable from human communication. Proceedings of the National Academy of Sciences, 123(42), e2531748123. https://doi.org/10.1073/pnas.2531748123

How many times has this paper been cited?

10 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Proceedings of the National Academy of Sciences: sign in and open the full text from this page.

Who wrote this paper?

Nicolò Pagan, Petter Törnberg, Christopher A. Bail, Christopher Barrie.

What is the DOI of this paper?

10.1073/pnas.2531748123, permanently at https://doi.org/10.1073/pnas.2531748123. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.