Review article · 2026 · Open access

Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology

Venkateshbabu Nagendrababu, Nicolás Dufey-Portilla, Sheela Balu Abraham, Thilla Sekar Vinothkumar, Giorgos N. Tzanetakis, Jelena Jaćimović, Aleksandar Jakovljević, Fernando Peña‐Bengoa, Marc García‐Font, Francesc Abella Sans, Clóvis Mariano Faggion, P. M. H. Dummer

International Endodontic Journal · Published 28 October 2026 · DOI 10.1111/iej.70297

Open PDFView at the publisherCite this paper

In brief

“Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology” is a 2026 review article by Venkateshbabu Nagendrababu and 11 co-authors, published in International Endodontic Journal. OpenAlex counts 0 citations as of 17 November 2026. It is open access: the full text is free to read at International Endodontic Journal.

Citations
0
References
126
Authors
12

Abstract

OBJECTIVE: To evaluate the level of agreement between ChatGPT-5.5 and human assessments when appraising the methodological quality and overall confidence of the results of systematic reviews (SRs) with meta-analysis (MA) in endodontology. METHODS: SRs with MA related to endodontology and published in English between 2021 and 2024 were included. The methodological quality of the SRs was appraised by three human experts using AMSTAR 2, and a consensus dataset was generated. ChatGPT-5.5 was applied to generate AMSTAR 2 item-level ratings and overall confidence classifications for each SR through three newly created accounts. Agreement between consolidated ChatGPT-5.5 ratings and human consensus was assessed for both overall confidence and each AMSTAR 2 item. Overall-confidence agreement was evaluated under the primary criterion (Partial Yes = Yes) and an alternative sensitivity criterion (Partial Yes = No). Direct agreement and linearly weighted Cohen's kappa were calculated, and complete 3/3 agreement was used to assess consistency across ChatGPT-5.5 accounts. RESULTS: One hundred SRs were included. Human consensus and ChatGPT-5.5 classified 47 and 85 SRs, respectively, as critically low. Under the primary criterion (Partial Yes = Yes), overall confidence matched in 52/100 SRs (52%; weighted κ = 0.182). Under the alternative criterion (Partial Yes = No), agreement increased to 83%, while weighted κ remained similar (κ = 0.173), indicating that direct agreement was sensitive to the handling of Partial Yes. Item-level agreement varied substantially. Overall confidence was identical across the three ChatGPT-5.5 accounts in 85 SRs, and global item-level 3/3 agreement was 86.12%. In the 100 sampled SRs, 15% met the criterion for reporting primary-study funding and 21% for justifying study designs. CONCLUSION: Overall-confidence direct agreement was sensitive to the handling of Partial Yes ratings, whereas chance-corrected agreement remained similar under both criteria. ChatGPT-5.5 assigned lower overall confidence classifications more frequently than human consensus, and agreement varied across AMSTAR 2 domains. Although outputs were frequently consistent across the three ChatGPT-5.5 accounts, between-account consistency did not ensure concordance with human judgements. The findings do not support the use of ChatGPT-5.5 as an independent AMSTAR 2 assessor; any exploratory application requires item-by-item verification by reviewers with methodological expertise.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Nagendrababu, V., Dufey-Portilla, N., Abraham, S. B., Vinothkumar, T. S., Tzanetakis, G. N., Jaćimović, J., Jakovljević, A., Peña‐Bengoa, F., García‐Font, M., Sans, F. A., Faggion, C. M., & Dummer, P. M. H. (2026). Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology. International Endodontic Journal. https://doi.org/10.1111/iej.70297

MLA 9

Nagendrababu, Venkateshbabu, et al. “Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology.” International Endodontic Journal, 2026. https://doi.org/10.1111/iej.70297.

Chicago

Nagendrababu, Venkateshbabu, Nicolás Dufey-Portilla, Sheela Balu Abraham, Thilla Sekar Vinothkumar, Giorgos N. Tzanetakis, Jelena Jaćimović, Aleksandar Jakovljević, et al. 2026. “Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology.” International Endodontic Journal. https://doi.org/10.1111/iej.70297.

Harvard

Nagendrababu, V. et al. (2026) ‘Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology’, International Endodontic Journal. doi:10.1111/iej.70297.

IEEE

V. Nagendrababu et al., “Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology,” International Endodontic Journal, 2026, doi: 10.1111/iej.70297.

BibTeX

@article{nagendrababu2026human,
  title     = {{Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology}},
  author    = {Nagendrababu, Venkateshbabu and Dufey-Portilla, Nicolás and Abraham, Sheela Balu and Vinothkumar, Thilla Sekar and Tzanetakis, Giorgos N. and Jaćimović, Jelena and Jakovljević, Aleksandar and Peña‐Bengoa, Fernando and García‐Font, Marc and Sans, Francesc Abella and Faggion, Clóvis Mariano and Dummer, P. M. H.},
  journal   = {International Endodontic Journal},
  year      = {2026},
  doi       = {10.1111/iej.70297},
  url       = {https://doi.org/10.1111/iej.70297}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews

  • large language models
  • AMSTAR 2
  • methodological quality assessment
  • systematic reviews
  • meta-analysis
  • reporting quality
  • Cohen's kappa
  • interobserver agreement

Work with this paper

Most-cited papers in Decision Sciences

All highly cited Decision Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Nagendrababu, V., Dufey-Portilla, N., Abraham, S. B., Vinothkumar, T. S., Tzanetakis, G. N., Jaćimović, J., Jakovljević, A., Peña‐Bengoa, F., García‐Font, M., Sans, F. A., Faggion, C. M., & Dummer, P. M. H. (2026). Human Consensus Versus Generative Artificial Intelligence When Assessing the Methodological Quality and Overall Confidence in the Results of Systematic Reviews With Meta‐Analysis in Endodontology. International Endodontic Journal. https://doi.org/10.1111/iej.70297

How many times has this paper been cited?

0 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at International Endodontic Journal: sign in and open the PDF from this page.

Who wrote this paper?

Venkateshbabu Nagendrababu, Nicolás Dufey-Portilla, Sheela Balu Abraham, Thilla Sekar Vinothkumar, Giorgos N. Tzanetakis, Jelena Jaćimović, Aleksandar Jakovljević, Fernando Peña‐Bengoa, Marc García‐Font, Francesc Abella Sans, Clóvis Mariano Faggion, P. M. H. Dummer.

What is the DOI of this paper?

10.1111/iej.70297, permanently at https://doi.org/10.1111/iej.70297. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.