Journal article · 2026 · Open access

Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies

Sulaiman Abubakar Musa

BMC Medical Informatics and Decision Making · Published 23 September 2026 · DOI 10.1186/s12911-026-03866-7

Read the free full textCite this paper

In brief

“Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies” is a 2026 journal article by Sulaiman Abubakar Musa, published in BMC Medical Informatics and Decision Making. OpenAlex counts 0 citations as of 15 November 2026. It is open access under a CC BY-NC-ND licence: the full text is free to read at BMC Medical Informatics and Decision Making.

Citations
0
Authors
1

Abstract

Artificial intelligence (AI) and machine-learning prediction models are increasingly used in clinical research, but their trustworthiness depends on transparent, complete, and reproducible reporting. Despite the introduction of the TRIPOD + AI guideline, manual assessment of adherence to its 27 items remains labour-intensive, and no validated automated tool for this purpose has previously been reported. To develop and rigorously validate a transparent, rule-based automated tool (TripodAI) for assessing adherence to the 27-item TRIPOD + AI reporting guideline in AI prediction-model manuscripts. TripodAI is an open-source, rule-based Python package that detects TRIPOD + AI reporting elements and issues a binary (Yes/No) adequacy judgement for each item. The reference standard was a 70-study subsample drawn from a parent meta-epidemiological census of 255 AI-based clinical prediction model studies, previously dual-extracted and adjudicated (inter-rater κ = 0.611, 84.5% raw agreement). Internal validation compared TripodAI against this manual reference standard across 1,890 item-level pairs; a further held-out external validation set drawn from the same parent census set of 70 studies (1,890 item-level pairs) was subsequently assessed. Agreement was quantified using percent agreement and Cohen’s κ; diagnostic performance (accuracy, sensitivity, specificity, precision, F1-score) was calculated using manual review as the reference standard. In internal validation, overall percent agreement was 86.0% and Cohen’s κ was 0.705 (95% CI: 0.639–0.771), indicating substantial agreement. Binary diagnostic performance (n = 1,766) showed accuracy of 0.881, sensitivity of 0.925, specificity of 0.777, precision of 0.906, and F1-score of 0.915. Performance was strongest for standardised methodological items (e.g., algorithm type specified, κ = 1.000; setting and recruitment, κ = 1.000) and weakest for data availability (κ = 0.227) and feature selection method (κ = 0.278). External validation on a held-out set of 70 studies drawn from the same parent census confirmed these findings: percent agreement was 85.1%, Cohen’s κ was 0.691 (95% CI: 0.627–0.756), accuracy was 0.860, sensitivity 0.903, specificity 0.759, precision 0.899, and F1-score 0.901 — demonstrating minimal attenuation from internal validation. TripodAI, a transparent rule-based automated extraction tool, achieved substantial agreement with independent manual review, with high sensitivity and acceptable specificity. While not yet suitable for fully autonomous use, particularly for data-availability statements, the tool performs reliably as a human-in-the-loop decision-support system for large-scale TRIPOD + AI adherence assessment in systematic reviews, meta-research, and journal editorial workflows. Not applicable. To our knowledge, no validated automated tool for scoring adherence to the 27-item TRIPOD + AI checklist existed prior to this study. TripodAI achieved substantial agreement with human manual review in both internal (κ = 0.705) and external (κ = 0.691) validation.…

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Musa, S. A. (2026). Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-026-03866-7

MLA 9

Musa, Sulaiman Abubakar. “Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies.” BMC Medical Informatics and Decision Making, 2026. https://doi.org/10.1186/s12911-026-03866-7.

Chicago

Musa, Sulaiman Abubakar. 2026. “Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies.” BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-026-03866-7.

Harvard

Musa, S. A. (2026) ‘Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies’, BMC Medical Informatics and Decision Making. doi:10.1186/s12911-026-03866-7.

IEEE

S. A. Musa, “Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies,” BMC Medical Informatics and Decision Making, 2026, doi: 10.1186/s12911-026-03866-7.

BibTeX

@article{musa2026development,
  title     = {{Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies}},
  author    = {Musa, Sulaiman Abubakar},
  journal   = {BMC Medical Informatics and Decision Making},
  year      = {2026},
  doi       = {10.1186/s12911-026-03866-7},
  url       = {https://doi.org/10.1186/s12911-026-03866-7}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews

  • TRIPOD+AI
  • reporting guidelines
  • artificial intelligence prediction models
  • machine learning
  • clinical prediction model
  • guideline adherence
  • data availability
  • feature selection

Work with this paper

Most-cited papers in Decision Sciences

All highly cited Decision Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Musa, S. A. (2026). Development and validation of an automated rule-based tool for assessing TRIPOD + AI reporting adherence in artificial intelligence prediction model studies. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-026-03866-7

How many times has this paper been cited?

0 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at BMC Medical Informatics and Decision Making.

Who wrote this paper?

Sulaiman Abubakar Musa.

What is the DOI of this paper?

10.1186/s12911-026-03866-7. The permanent link is https://doi.org/10.1186/s12911-026-03866-7.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.