Review article · 2026 · Open access
Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology
International Endodontic Journal · Published 28 October 2026 · DOI 10.1111/iej.70297
In brief
“Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology” is a 2026 review article by Venkateshbabu Nagendrababu and 11 co-authors, published in International Endodontic Journal. OpenAlex counts 0 citations as of 17 November 2026. It is open access: the full text is free to read at International Endodontic Journal.
- Citations
- 0
- References
- 126
- Authors
- 12
Abstract
OBJECTIVE: To evaluate the level of agreement between ChatGPT-5.5 and human assessments when appraising the methodological quality and overall confidence of the results of systematic reviews (SRs) with meta-analysis (MA) in endodontology. METHODS: SRs with MA related to endodontology and published in English between 2021 and 2024 were included. The methodological quality of the SRs was appraised by three human experts using AMSTAR 2, and a consensus dataset was generated. ChatGPT-5.5 was applied to generate AMSTAR 2 item-level ratings and overall confidence classifications for each SR through three newly created accounts. Agreement between consolidated ChatGPT-5.5 ratings and human consensus was assessed for both overall confidence and each AMSTAR 2 item. Overall-confidence agreement was evaluated under the primary criterion (Partial Yes = Yes) and an alternative sensitivity criterion (Partial Yes = No). Direct agreement and linearly weighted Cohen's kappa were calculated, and complete 3/3 agreement was used to assess consistency across ChatGPT-5.5 accounts. RESULTS: One hundred SRs were included. Human consensus and ChatGPT-5.5 classified 47 and 85 SRs, respectively, as critically low. Under the primary criterion (Partial Yes = Yes), overall confidence matched in 52/100 SRs (52%; weighted κ = 0.182). Under the alternative criterion (Partial Yes = No), agreement increased to 83%, while weighted κ remained similar (κ = 0.173), indicating that direct agreement was sensitive to the handling of Partial Yes. Item-level agreement varied substantially. Overall confidence was identical across the three ChatGPT-5.5 accounts in 85 SRs, and global item-level 3/3 agreement was 86.12%. In the 100 sampled SRs, 15% met the criterion for reporting primary-study funding and 21% for justifying study designs. CONCLUSION: Overall-confidence direct agreement was sensitive to the handling of Partial Yes ratings, whereas chance-corrected agreement remained similar under both criteria. ChatGPT-5.5 assigned lower overall confidence classifications more frequently than human consensus, and agreement varied across AMSTAR 2 domains. Although outputs were frequently consistent across the three ChatGPT-5.5 accounts, between-account consistency did not ensure concordance with human judgements. The findings do not support the use of ChatGPT-5.5 as an independent AMSTAR 2 assessor; any exploratory application requires item-by-item verification by reviewers with methodological expertise.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Nagendrababu, V., Dufey-Portilla, N., Abraham, S. B., Vinothkumar, T. S., Tzanetakis, G. N., Jaćimović, J., Jakovljević, A., Peña‐Bengoa, F., García‐Font, M., Sans, F. A., Faggion, C. M., & Dummer, P. M. H. (2026). Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology. International Endodontic Journal. https://doi.org/10.1111/iej.70297
MLA 9
Nagendrababu, Venkateshbabu, et al. “Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology.” International Endodontic Journal, 2026. https://doi.org/10.1111/iej.70297.
Chicago
Nagendrababu, Venkateshbabu, Nicolás Dufey-Portilla, Sheela Balu Abraham, Thilla Sekar Vinothkumar, Giorgos N. Tzanetakis, Jelena Jaćimović, Aleksandar Jakovljević, et al. 2026. “Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology.” International Endodontic Journal. https://doi.org/10.1111/iej.70297.
Harvard
Nagendrababu, V. et al. (2026) ‘Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology’, International Endodontic Journal. doi:10.1111/iej.70297.
IEEE
V. Nagendrababu et al., “Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology,” International Endodontic Journal, 2026, doi: 10.1111/iej.70297.
BibTeX
@article{nagendrababu2026human,
title = {{Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology}},
author = {Nagendrababu, Venkateshbabu and Dufey-Portilla, Nicolás and Abraham, Sheela Balu and Vinothkumar, Thilla Sekar and Tzanetakis, Giorgos N. and Jaćimović, Jelena and Jakovljević, Aleksandar and Peña‐Bengoa, Fernando and García‐Font, Marc and Sans, Francesc Abella and Faggion, Clóvis Mariano and Dummer, P. M. H.},
journal = {International Endodontic Journal},
year = {2026},
doi = {10.1111/iej.70297},
url = {https://doi.org/10.1111/iej.70297}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- large language models
- AMSTAR 2
- methodological quality assessment
- systematic reviews
- meta-analysis
- reporting quality
- Cohen's kappa
- interobserver agreement
Work with this paper
Most-cited papers in Decision Sciences
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsMatthew James Page, Joanne E. McKenzie, Patrick M. Bossuyt and 23 more · BMJ · 2021 · 144,919 citations
- Evaluating Structural Equation Models with Unobservable Variables and Measurement ErrorClaes Fornell, David F. Larcker · Journal of Marketing Research · 1981 · 118,336 citations
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesLi‐tze Hu, Peter M. Bentler · Structural Equation Modeling A Multidisciplinary Journal · 1999 · 117,461 citations
- Common method biases in behavioral research: A critical review of the literature and recommended remedies.Philip M. Podsakoff, SCOTT B. MACKENZIE, Jeong Yeon Lee and 1 more · Journal of Applied Psychology · 2003 · 89,248 citations
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statementDavid Moher, Alessandro Liberati, Jennifer Tetzlaff and 2 more · BMJ · 2009 · 85,524 citations
Questions about this paper
How do I cite this paper in APA style?
Nagendrababu, V., Dufey-Portilla, N., Abraham, S. B., Vinothkumar, T. S., Tzanetakis, G. N., Jaćimović, J., Jakovljević, A., Peña‐Bengoa, F., García‐Font, M., Sans, F. A., Faggion, C. M., & Dummer, P. M. H. (2026). Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology. International Endodontic Journal. https://doi.org/10.1111/iej.70297
How many times has this paper been cited?
0 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at International Endodontic Journal: sign in and open the PDF from this page.
Who wrote this paper?
Venkateshbabu Nagendrababu, Nicolás Dufey-Portilla, Sheela Balu Abraham, Thilla Sekar Vinothkumar, Giorgos N. Tzanetakis, Jelena Jaćimović, Aleksandar Jakovljević, Fernando Peña‐Bengoa, Marc García‐Font, Francesc Abella Sans, Clóvis Mariano Faggion, P. M. H. Dummer.
What is the DOI of this paper?
10.1111/iej.70297, permanently at https://doi.org/10.1111/iej.70297. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.