Preprint · 2025 · Open access

The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis

Katherine Mok, Daniel Z. Q. Gan, Annabel Burnside, Caroline X. Gao, Kate Filia

PsyArXiv (OSF Preprints) · Published 11 September 2025 · DOI 10.31234/osf.io/ngb65_v1

Open full textCite this paper

In brief

“The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis” is a 2025 preprint by Katherine Mok and 4 co-authors, published in PsyArXiv (OSF Preprints). OpenAlex counts 0 citations as of 18 November 2026. It is open access under a CC BY licence: the full text is free to read at PsyArXiv (OSF Preprints).

Citations
0
Authors
5

Abstract

ObjectivesArtificial Intelligence (AI) tools present an opportunity to expedite the typically lengthy process of systematic reviews and meta-analyses, but more evidence is required on their performance in practice. This paper examined the use of ASReview and ChatGPT for screening, data extraction, and quality ratings compared against a traditional systematic review to explore potential efficiency, accuracy of information, and consistency of judgements compared to human reviewers. MethodsThree screening simulations were conducted using ASReview with different amounts of training data (1, 3, or 5 irrelevant/relevant records). A standardised set of prompts were developed and uploaded to ChatGPT for data extraction and coded for accuracy against the primary studies. For quality ratings, Cochrane’s guidance for the Risk of Bias 2.0 and Risk of Bias in Non-Randomised Studies of Interventions tools were provided to ChatGPT, with their respective templates.ResultsIn all simulations ASReview prioritised relevant studies within the first 800 records (approximately 17% of the dataset). When extracting, ChatGPT sometimes omitted information, though further detail was provided with additional prompting. Few instances of inaccurate information were observed. Consistency of quality ratings was low to moderate, depending on the domain of bias. ConclusionsAI tools can reduce the time required for screening through effective prioritisation of relevant articles and might also support data extraction by quickly locating relevant information in a manuscript. They should, however, be approached with caution for more complex tasks (e.g., quality ratings). In any case, the use of AI requires careful testing and validation of outputs.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Mok, K., Gan, D. Z. Q., Burnside, A., Gao, C. X., & Filia, K. (2025). The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis. PsyArXiv (OSF Preprints). https://doi.org/10.31234/osf.io/ngb65_v1

MLA 9

Mok, Katherine, et al. “The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis.” PsyArXiv (OSF Preprints), 2025. https://doi.org/10.31234/osf.io/ngb65_v1.

Chicago

Mok, Katherine, Daniel Z. Q. Gan, Annabel Burnside, Caroline X. Gao, and Kate Filia. 2025. “The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis.” PsyArXiv (OSF Preprints). https://doi.org/10.31234/osf.io/ngb65_v1.

Harvard

Mok, K. et al. (2025) ‘The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis’, PsyArXiv (OSF Preprints). doi:10.31234/osf.io/ngb65_v1.

IEEE

K. Mok, D. Z. Q. Gan, A. Burnside, C. X. Gao, and K. Filia, “The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis,” PsyArXiv (OSF Preprints), 2025, doi: 10.31234/osf.io/ngb65_v1.

BibTeX

@misc{mok2025efficiency,
  title     = {{The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis}},
  author    = {Mok, Katherine and Gan, Daniel Z. Q. and Burnside, Annabel and Gao, Caroline X. and Filia, Kate},
  howpublished = {PsyArXiv (OSF Preprints)},
  year      = {2025},
  doi       = {10.31234/osf.io/ngb65_v1},
  url       = {https://doi.org/10.31234/osf.io/ngb65_v1}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews

  • ASReview
  • ChatGPT
  • study screening
  • data extraction
  • risk of bias assessment
  • prompt engineering
  • AI training data
  • meta-analysis

Work with this paper

Most-cited papers in Decision Sciences

All highly cited Decision Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Mok, K., Gan, D. Z. Q., Burnside, A., Gao, C. X., & Filia, K. (2025). The efficiency and accuracy of Artificial Intelligence in conducting systematic reviews: A single case analysis. PsyArXiv (OSF Preprints). https://doi.org/10.31234/osf.io/ngb65_v1

How many times has this paper been cited?

0 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at PsyArXiv (OSF Preprints): sign in and open the full text from this page.

Who wrote this paper?

Katherine Mok, Daniel Z. Q. Gan, Annabel Burnside, Caroline X. Gao, Kate Filia.

What is the DOI of this paper?

10.31234/osf.io/ngb65_v1, permanently at https://doi.org/10.31234/osf.io/ngb65_v1. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.