Journal article · 2026 · Open access

Evaluating the statistical realism of LLM-generated social science data

Yueqi Xie, Lemeng Liang, Shuzhen Li, Yifu Lu, Zhiwen Xiao, Mengdi Shi, Junming Huang, Mengdi Wang, Yu Xie

Proceedings of the National Academy of Sciences, 123(19), e2538145123 · Published 8 May 2026 · DOI 10.1073/pnas.2538145123

Open full textCite this paper

In brief

“Evaluating the statistical realism of LLM-generated social science data” is a 2026 journal article by Yueqi Xie and 8 co-authors, published in Proceedings of the National Academy of Sciences. OpenAlex counts 39 citations as of 18 November 2026. It is open access under a CC BY-NC-ND licence: the full text is free to read at Proceedings of the National Academy of Sciences.

Citations
39
Most cited in
2026 (38)
Field-weighted impact
41.08
References
48
Authors
9

Abstract

Large language models (LLMs) hold great promise for generating social science data, potentially expanding the methodological toolkit of quantitative social research. Prior studies have primarily focused on individual-level predictability or behavioral plausibility of LLM-generated data. We propose a framework for assessing the validity of LLM-generated data by returning to the foundational principles of survey research in the social sciences. Just as surveys based on representative samples yield statistics that approximate the corresponding statistical moments of the target population, assessment should center on the ability of LLM-generated data to reproduce real-world, population-level statistical patterns. We introduce SSDataBench, a systematic benchmark designed to evaluate population-level statistical realism in LLM-generated social science data. The benchmark assesses five types of statistical patterns central to social research: univariate distributions, bivariate associations, multivariate outcome predictions, life event sequence distributions, and associations between life event sequences and covariates. We illustrate SSDataBench using four longitudinal datasets and three cross-sectional datasets spanning six major social domains: demographics, socioeconomic status, marriage, health, abilities, and attitudes. Our study reveals representational limitations in current LLMs under sparse conditioning settings, manifested in a pronounced tendency to compress real-world heterogeneity into simplified typological structures. Finally, we outline a roadmap toward improved statistical realism and report preliminary results indicating that domain-specific training can enhance population-level realism.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Xie, Y., Liang, L., Li, S., Lu, Y., Xiao, Z., Shi, M., Huang, J., Wang, M., & Xie, Y. (2026). Evaluating the statistical realism of LLM-generated social science data. Proceedings of the National Academy of Sciences, 123(19), e2538145123. https://doi.org/10.1073/pnas.2538145123

MLA 9

Xie, Yueqi, et al. “Evaluating the statistical realism of LLM-generated social science data.” Proceedings of the National Academy of Sciences, vol. 123, no. 19, 2026, p. e2538145123. https://doi.org/10.1073/pnas.2538145123.

Chicago

Xie, Yueqi, Lemeng Liang, Shuzhen Li, Yifu Lu, Zhiwen Xiao, Mengdi Shi, Junming Huang, Mengdi Wang, and Yu Xie. 2026. “Evaluating the statistical realism of LLM-generated social science data.” Proceedings of the National Academy of Sciences 123 (19): e2538145123. https://doi.org/10.1073/pnas.2538145123.

Harvard

Xie, Y. et al. (2026) ‘Evaluating the statistical realism of LLM-generated social science data’, Proceedings of the National Academy of Sciences, 123(19), p. e2538145123. doi:10.1073/pnas.2538145123.

IEEE

Y. Xie et al., “Evaluating the statistical realism of LLM-generated social science data,” Proceedings of the National Academy of Sciences, vol. 123, no. 19, p. e2538145123, 2026, doi: 10.1073/pnas.2538145123.

BibTeX

@article{xie2026evaluating,
  title     = {{Evaluating the statistical realism of LLM-generated social science data}},
  author    = {Xie, Yueqi and Liang, Lemeng and Li, Shuzhen and Lu, Yifu and Xiao, Zhiwen and Shi, Mengdi and Huang, Junming and Wang, Mengdi and Xie, Yu},
  journal   = {Proceedings of the National Academy of Sciences},
  year      = {2026},
  volume    = {123},
  number    = {19},
  pages     = {e2538145123},
  doi       = {10.1073/pnas.2538145123},
  url       = {https://doi.org/10.1073/pnas.2538145123}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI

  • large language models
  • LLM-generated data
  • benchmarking
  • univariate distributions
  • bivariate relationships
  • longitudinal data
  • domain-specific training
  • data heterogeneity

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Xie, Y., Liang, L., Li, S., Lu, Y., Xiao, Z., Shi, M., Huang, J., Wang, M., & Xie, Y. (2026). Evaluating the statistical realism of LLM-generated social science data. Proceedings of the National Academy of Sciences, 123(19), e2538145123. https://doi.org/10.1073/pnas.2538145123

How many times has this paper been cited?

39 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Proceedings of the National Academy of Sciences: sign in and open the full text from this page.

Who wrote this paper?

Yueqi Xie, Lemeng Liang, Shuzhen Li, Yifu Lu, Zhiwen Xiao, Mengdi Shi, Junming Huang, Mengdi Wang, Yu Xie.

What is the DOI of this paper?

10.1073/pnas.2538145123, permanently at https://doi.org/10.1073/pnas.2538145123. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.