Preprint · 2025 · Open access
The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis
PsyArXiv (OSF Preprints) · Published 11 September 2025 · DOI 10.31234/osf.io/ngb65_v1
In brief
“The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis” is a 2025 preprint by Katherine Mok and 4 co-authors, published in PsyArXiv (OSF Preprints). OpenAlex counts 0 citations as of 18 November 2026. It is open access under a CC BY licence: the full text is free to read at PsyArXiv (OSF Preprints).
- Citations
- 0
- Authors
- 5
Abstract
ObjectivesArtificial Intelligence (AI) tools present an opportunity to expedite the typically lengthy process of systematic reviews and meta-analyses, but more evidence is required on their performance in practice. This paper examined the use of ASReview and ChatGPT for screening, data extraction, and quality ratings compared against a traditional systematic review to explore potential efficiency, accuracy of information, and consistency of judgements compared to human reviewers. MethodsThree screening simulations were conducted using ASReview with different amounts of training data (1, 3, or 5 irrelevant/relevant records). A standardised set of prompts were developed and uploaded to ChatGPT for data extraction and coded for accuracy against the primary studies. For quality ratings, Cochrane’s guidance for the Risk of Bias 2.0 and Risk of Bias in Non-Randomised Studies of Interventions tools were provided to ChatGPT, with their respective templates.ResultsIn all simulations ASReview prioritised relevant studies within the first 800 records (approximately 17% of the dataset). When extracting, ChatGPT sometimes omitted information, though further detail was provided with additional prompting. Few instances of inaccurate information were observed. Consistency of quality ratings was low to moderate, depending on the domain of bias. ConclusionsAI tools can reduce the time required for screening through effective prioritisation of relevant articles and might also support data extraction by quickly locating relevant information in a manuscript. They should, however, be approached with caution for more complex tasks (e.g., quality ratings). In any case, the use of AI requires careful testing and validation of outputs.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Mok, K., Gan, D. Z. Q., Burnside, A., Gao, C. X., & Filia, K. (2025). The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis. PsyArXiv (OSF Preprints). https://doi.org/10.31234/osf.io/ngb65_v1
MLA 9
Mok, Katherine, et al. “The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis.” PsyArXiv (OSF Preprints), 2025. https://doi.org/10.31234/osf.io/ngb65_v1.
Chicago
Mok, Katherine, Daniel Z. Q. Gan, Annabel Burnside, Caroline X. Gao, and Kate Filia. 2025. “The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis.” PsyArXiv (OSF Preprints). https://doi.org/10.31234/osf.io/ngb65_v1.
Harvard
Mok, K. et al. (2025) ‘The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis’, PsyArXiv (OSF Preprints). doi:10.31234/osf.io/ngb65_v1.
IEEE
K. Mok, D. Z. Q. Gan, A. Burnside, C. X. Gao, and K. Filia, “The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis,” PsyArXiv (OSF Preprints), 2025, doi: 10.31234/osf.io/ngb65_v1.
BibTeX
@misc{mok2025efficiency,
title = {{The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis}},
author = {Mok, Katherine and Gan, Daniel Z. Q. and Burnside, Annabel and Gao, Caroline X. and Filia, Kate},
howpublished = {PsyArXiv (OSF Preprints)},
year = {2025},
doi = {10.31234/osf.io/ngb65_v1},
url = {https://doi.org/10.31234/osf.io/ngb65_v1}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- ASReview
- ChatGPT
- study screening
- data extraction
- risk of bias assessment
- prompt engineering
- AI training data
- meta-analysis
Work with this paper
Most-cited papers in Decision Sciences
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsMatthew James Page, Joanne E. McKenzie, Patrick M. Bossuyt and 23 more · BMJ · 2021 · 144,919 citations
- Evaluating Structural Equation Models with Unobservable Variables and Measurement ErrorClaes Fornell, David F. Larcker · Journal of Marketing Research · 1981 · 118,336 citations
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesLi‐tze Hu, Peter M. Bentler · Structural Equation Modeling A Multidisciplinary Journal · 1999 · 117,461 citations
- Common method biases in behavioral research: A critical review of the literature and recommended remedies.Philip M. Podsakoff, SCOTT B. MACKENZIE, Jeong Yeon Lee and 1 more · Journal of Applied Psychology · 2003 · 89,248 citations
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statementDavid Moher, Alessandro Liberati, Jennifer Tetzlaff and 2 more · BMJ · 2009 · 85,524 citations
Questions about this paper
How do I cite this paper in APA style?
Mok, K., Gan, D. Z. Q., Burnside, A., Gao, C. X., & Filia, K. (2025). The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis. PsyArXiv (OSF Preprints). https://doi.org/10.31234/osf.io/ngb65_v1
How many times has this paper been cited?
0 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at PsyArXiv (OSF Preprints): sign in and open the full text from this page.
Who wrote this paper?
Katherine Mok, Daniel Z. Q. Gan, Annabel Burnside, Caroline X. Gao, Kate Filia.
What is the DOI of this paper?
10.31234/osf.io/ngb65_v1, permanently at https://doi.org/10.31234/osf.io/ngb65_v1. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.