Journal article · 2026 · Open access
Evaluating the statistical realism of LLM-generated social science data
Proceedings of the National Academy of Sciences, 123(19), e2538145123 · Published 8 May 2026 · DOI 10.1073/pnas.2538145123
In brief
“Evaluating the statistical realism of LLM-generated social science data” is a 2026 journal article by Yueqi Xie and 8 co-authors, published in Proceedings of the National Academy of Sciences. OpenAlex counts 39 citations as of 18 November 2026. It is open access under a CC BY-NC-ND licence: the full text is free to read at Proceedings of the National Academy of Sciences.
- Citations
- 39
- Most cited in
- 2026 (38)
- Field-weighted impact
- 41.08
- References
- 48
- Authors
- 9
Abstract
Large language models (LLMs) hold great promise for generating social science data, potentially expanding the methodological toolkit of quantitative social research. Prior studies have primarily focused on individual-level predictability or behavioral plausibility of LLM-generated data. We propose a framework for assessing the validity of LLM-generated data by returning to the foundational principles of survey research in the social sciences. Just as surveys based on representative samples yield statistics that approximate the corresponding statistical moments of the target population, assessment should center on the ability of LLM-generated data to reproduce real-world, population-level statistical patterns. We introduce SSDataBench, a systematic benchmark designed to evaluate population-level statistical realism in LLM-generated social science data. The benchmark assesses five types of statistical patterns central to social research: univariate distributions, bivariate associations, multivariate outcome predictions, life event sequence distributions, and associations between life event sequences and covariates. We illustrate SSDataBench using four longitudinal datasets and three cross-sectional datasets spanning six major social domains: demographics, socioeconomic status, marriage, health, abilities, and attitudes. Our study reveals representational limitations in current LLMs under sparse conditioning settings, manifested in a pronounced tendency to compress real-world heterogeneity into simplified typological structures. Finally, we outline a roadmap toward improved statistical realism and report preliminary results indicating that domain-specific training can enhance population-level realism.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Xie, Y., Liang, L., Li, S., Lu, Y., Xiao, Z., Shi, M., Huang, J., Wang, M., & Xie, Y. (2026). Evaluating the statistical realism of LLM-generated social science data. Proceedings of the National Academy of Sciences, 123(19), e2538145123. https://doi.org/10.1073/pnas.2538145123
MLA 9
Xie, Yueqi, et al. “Evaluating the statistical realism of LLM-generated social science data.” Proceedings of the National Academy of Sciences, vol. 123, no. 19, 2026, p. e2538145123. https://doi.org/10.1073/pnas.2538145123.
Chicago
Xie, Yueqi, Lemeng Liang, Shuzhen Li, Yifu Lu, Zhiwen Xiao, Mengdi Shi, Junming Huang, Mengdi Wang, and Yu Xie. 2026. “Evaluating the statistical realism of LLM-generated social science data.” Proceedings of the National Academy of Sciences 123 (19): e2538145123. https://doi.org/10.1073/pnas.2538145123.
Harvard
Xie, Y. et al. (2026) ‘Evaluating the statistical realism of LLM-generated social science data’, Proceedings of the National Academy of Sciences, 123(19), p. e2538145123. doi:10.1073/pnas.2538145123.
IEEE
Y. Xie et al., “Evaluating the statistical realism of LLM-generated social science data,” Proceedings of the National Academy of Sciences, vol. 123, no. 19, p. e2538145123, 2026, doi: 10.1073/pnas.2538145123.
BibTeX
@article{xie2026evaluating,
title = {{Evaluating the statistical realism of LLM-generated social science data}},
author = {Xie, Yueqi and Liang, Lemeng and Li, Shuzhen and Lu, Yifu and Xiao, Zhiwen and Shi, Mengdi and Huang, Junming and Wang, Mengdi and Xie, Yu},
journal = {Proceedings of the National Academy of Sciences},
year = {2026},
volume = {123},
number = {19},
pages = {e2538145123},
doi = {10.1073/pnas.2538145123},
url = {https://doi.org/10.1073/pnas.2538145123}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- large language models
- LLM-generated data
- benchmarking
- univariate distributions
- bivariate relationships
- longitudinal data
- domain-specific training
- data heterogeneity
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Xie, Y., Liang, L., Li, S., Lu, Y., Xiao, Z., Shi, M., Huang, J., Wang, M., & Xie, Y. (2026). Evaluating the statistical realism of LLM-generated social science data. Proceedings of the National Academy of Sciences, 123(19), e2538145123. https://doi.org/10.1073/pnas.2538145123
How many times has this paper been cited?
39 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Proceedings of the National Academy of Sciences: sign in and open the full text from this page.
Who wrote this paper?
Yueqi Xie, Lemeng Liang, Shuzhen Li, Yifu Lu, Zhiwen Xiao, Mengdi Shi, Junming Huang, Mengdi Wang, Yu Xie.
What is the DOI of this paper?
10.1073/pnas.2538145123, permanently at https://doi.org/10.1073/pnas.2538145123. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.