Journal article · 2026 · Open access
Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies
BMC Medical Informatics and Decision Making · Published 23 September 2026 · DOI 10.1186/s12911-026-03866-7
In brief
“Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies” is a 2026 journal article by Sulaiman Abubakar Musa, published in BMC Medical Informatics and Decision Making. OpenAlex counts 0 citations as of 15 November 2026. It is open access under a CC BY-NC-ND licence: the full text is free to read at BMC Medical Informatics and Decision Making.
- Citations
- 0
- Authors
- 1
Abstract
Artificial intelligence (AI) and machine-learning prediction models are increasingly used in clinical research, but their trustworthiness depends on transparent, complete, and reproducible reporting. Despite the introduction of the TRIPOD + AI guideline, manual assessment of adherence to its 27 items remains labour-intensive, and no validated automated tool for this purpose has previously been reported. To develop and rigorously validate a transparent, rule-based automated tool (TripodAI) for assessing adherence to the 27-item TRIPOD + AI reporting guideline in AI prediction-model manuscripts. TripodAI is an open-source, rule-based Python package that detects TRIPOD + AI reporting elements and issues a binary (Yes/No) adequacy judgement for each item. The reference standard was a 70-study subsample drawn from a parent meta-epidemiological census of 255 AI-based clinical prediction model studies, previously dual-extracted and adjudicated (inter-rater κ = 0.611, 84.5% raw agreement). Internal validation compared TripodAI against this manual reference standard across 1,890 item-level pairs; a further held-out external validation set drawn from the same parent census set of 70 studies (1,890 item-level pairs) was subsequently assessed. Agreement was quantified using percent agreement and Cohen’s κ; diagnostic performance (accuracy, sensitivity, specificity, precision, F1-score) was calculated using manual review as the reference standard. In internal validation, overall percent agreement was 86.0% and Cohen’s κ was 0.705 (95% CI: 0.639–0.771), indicating substantial agreement. Binary diagnostic performance (n = 1,766) showed accuracy of 0.881, sensitivity of 0.925, specificity of 0.777, precision of 0.906, and F1-score of 0.915. Performance was strongest for standardised methodological items (e.g., algorithm type specified, κ = 1.000; setting and recruitment, κ = 1.000) and weakest for data availability (κ = 0.227) and feature selection method (κ = 0.278). External validation on a held-out set of 70 studies drawn from the same parent census confirmed these findings: percent agreement was 85.1%, Cohen’s κ was 0.691 (95% CI: 0.627–0.756), accuracy was 0.860, sensitivity 0.903, specificity 0.759, precision 0.899, and F1-score 0.901 — demonstrating minimal attenuation from internal validation. TripodAI, a transparent rule-based automated extraction tool, achieved substantial agreement with independent manual review, with high sensitivity and acceptable specificity. While not yet suitable for fully autonomous use, particularly for data-availability statements, the tool performs reliably as a human-in-the-loop decision-support system for large-scale TRIPOD + AI adherence assessment in systematic reviews, meta-research, and journal editorial workflows. Not applicable. To our knowledge, no validated automated tool for scoring adherence to the 27-item TRIPOD + AI checklist existed prior to this study. TripodAI achieved substantial agreement with human manual review in both internal (κ = 0.705) and external (κ = 0.691) validation.…
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Musa, S. A. (2026). Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-026-03866-7
MLA 9
Musa, Sulaiman Abubakar. “Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies.” BMC Medical Informatics and Decision Making, 2026. https://doi.org/10.1186/s12911-026-03866-7.
Chicago
Musa, Sulaiman Abubakar. 2026. “Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies.” BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-026-03866-7.
Harvard
Musa, S. A. (2026) ‘Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies’, BMC Medical Informatics and Decision Making. doi:10.1186/s12911-026-03866-7.
IEEE
S. A. Musa, “Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies,” BMC Medical Informatics and Decision Making, 2026, doi: 10.1186/s12911-026-03866-7.
BibTeX
@article{musa2026development,
title = {{Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies}},
author = {Musa, Sulaiman Abubakar},
journal = {BMC Medical Informatics and Decision Making},
year = {2026},
doi = {10.1186/s12911-026-03866-7},
url = {https://doi.org/10.1186/s12911-026-03866-7}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- TRIPOD+AI
- reporting guidelines
- artificial intelligence prediction models
- machine learning
- clinical prediction model
- guideline adherence
- data availability
- feature selection
Work with this paper
Most-cited papers in Decision Sciences
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsMatthew James Page, Joanne E. McKenzie, Patrick M. Bossuyt and 23 more · BMJ · 2021 · 144,919 citations
- Evaluating Structural Equation Models with Unobservable Variables and Measurement ErrorClaes Fornell, David F. Larcker · Journal of Marketing Research · 1981 · 118,336 citations
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesLi‐tze Hu, Peter M. Bentler · Structural Equation Modeling A Multidisciplinary Journal · 1999 · 117,461 citations
- Common method biases in behavioral research: A critical review of the literature and recommended remedies.Philip M. Podsakoff, SCOTT B. MACKENZIE, Jeong Yeon Lee and 1 more · Journal of Applied Psychology · 2003 · 89,248 citations
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statementDavid Moher, Alessandro Liberati, Jennifer Tetzlaff and 2 more · BMJ · 2009 · 85,524 citations
Questions about this paper
How do I cite this paper in APA style?
Musa, S. A. (2026). Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-026-03866-7
How many times has this paper been cited?
0 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at BMC Medical Informatics and Decision Making.
Who wrote this paper?
Sulaiman Abubakar Musa.
What is the DOI of this paper?
10.1186/s12911-026-03866-7. The permanent link is https://doi.org/10.1186/s12911-026-03866-7.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.