Paper · 2026 · Open access
How accurate is accurate enough? A systematic review and meta-analysis of data-extraction errors by large language models and human reviewers, with a simulation of their consequences for Cochrane meta-analyses
Open Science Framework · Published 27 September 2026 · DOI 10.17605/osf.io/h8rqm
In brief
“How accurate is accurate enough? A systematic review and meta-analysis of data-extraction errors by large language models and human reviewers, with a simulation of their consequences for Cochrane meta-analyses” is a 2026 paper by Chuan Yin, published in Open Science Framework. OpenAlex counts 0 citations as of 18 November 2026. It is open access under a CC BY licence: the full text is free to read at Open Science Framework.
- Citations
- 0
- Authors
- 1
Abstract
Large language models (LLMs) are increasingly used to extract data for systematic reviews. Published evaluations mostly report task-level accuracy, which does not tell users how likely extraction errors are to change the conclusions of a meta-analysis. We will (1) systematically review and meta-analyse evaluations of LLM data extraction from research reports, estimating the distribution and type spectrum of extraction errors; (2) synthesise studies of human extraction error as a comparator; and (3) propagate the estimated error distributions through real Cochrane meta-analyses by simulation, translating task-level accuracy into the probability that meta-analytic conclusions change.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Yin, C. (2026). How accurate is accurate enough? A systematic review and meta-analysis of data-extraction errors by large language models and human reviewers, with a simulation of their consequences for Cochrane meta-analyses. Open Science Framework. https://doi.org/10.17605/osf.io/h8rqm
MLA 9
Yin, Chuan. “How accurate is accurate enough? A systematic review and meta-analysis of data-extraction errors by large language models and human reviewers, with a simulation of their consequences for Cochrane meta-analyses.” Open Science Framework, 2026. https://doi.org/10.17605/osf.io/h8rqm.
Chicago
Yin, Chuan. 2026. “How accurate is accurate enough? A systematic review and meta-analysis of data-extraction errors by large language models and human reviewers, with a simulation of their consequences for Cochrane meta-analyses.” Open Science Framework. https://doi.org/10.17605/osf.io/h8rqm.
Harvard
Yin, C. (2026) ‘How accurate is accurate enough? A systematic review and meta-analysis of data-extraction errors by large language models and human reviewers, with a simulation of their consequences for Cochrane meta-analyses’, Open Science Framework. doi:10.17605/osf.io/h8rqm.
IEEE
C. Yin, “How accurate is accurate enough? A systematic review and meta-analysis of data-extraction errors by large language models and human reviewers, with a simulation of their consequences for Cochrane meta-analyses,” Open Science Framework, 2026, doi: 10.17605/osf.io/h8rqm.
BibTeX
@article{yin2026accurate,
title = {{How accurate is accurate enough? A systematic review and meta-analysis of data-extraction errors by large language models and human reviewers, with a simulation of their consequences for Cochrane meta-analyses}},
author = {Yin, Chuan},
journal = {Open Science Framework},
year = {2026},
doi = {10.17605/osf.io/h8rqm},
url = {https://doi.org/10.17605/osf.io/h8rqm}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- large language models
- data extraction errors
- systematic reviews
- error propagation
Work with this paper
Most-cited papers in Decision Sciences
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsMatthew James Page, Joanne E. McKenzie, Patrick M. Bossuyt and 23 more · BMJ · 2021 · 144,919 citations
- Evaluating Structural Equation Models with Unobservable Variables and Measurement ErrorClaes Fornell, David F. Larcker · Journal of Marketing Research · 1981 · 118,336 citations
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesLi‐tze Hu, Peter M. Bentler · Structural Equation Modeling A Multidisciplinary Journal · 1999 · 117,461 citations
- Common method biases in behavioral research: A critical review of the literature and recommended remedies.Philip M. Podsakoff, SCOTT B. MACKENZIE, Jeong Yeon Lee and 1 more · Journal of Applied Psychology · 2003 · 89,248 citations
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statementDavid Moher, Alessandro Liberati, Jennifer Tetzlaff and 2 more · BMJ · 2009 · 85,524 citations
Questions about this paper
How do I cite this paper in APA style?
Yin, C. (2026). How accurate is accurate enough? A systematic review and meta-analysis of data-extraction errors by large language models and human reviewers, with a simulation of their consequences for Cochrane meta-analyses. Open Science Framework. https://doi.org/10.17605/osf.io/h8rqm
How many times has this paper been cited?
0 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Open Science Framework: sign in and open the full text from this page.
Who wrote this paper?
Chuan Yin.
What is the DOI of this paper?
10.17605/osf.io/h8rqm, permanently at https://doi.org/10.17605/osf.io/h8rqm. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.