Review article · 2020 · Open access
Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques
BMJ Open, 10(11), e038832 · Published 1 November 2020 · DOI 10.1136/bmjopen-2020-038832
In brief
“Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques” is a 2020 review article by Constanza L. Andaur Navarro and 10 co-authors, published in BMJ Open. OpenAlex counts 88 citations as of 15 November 2026, which puts it in the top 2% of works published in 2020 by citations. It is open access under a CC BY licence: the full text is free to read at BMJ Open.
- Citations
- 88
- Most cited in
- 2022 (31)
- Field-weighted impact
- 2.00
- References
- 38
- Authors
- 11
Abstract
INTRODUCTION: Studies addressing the development and/or validation of diagnostic and prognostic prediction models are abundant in most clinical domains. Systematic reviews have shown that the methodological and reporting quality of prediction model studies is suboptimal. Due to the increasing availability of larger, routinely collected and complex medical data, and the rising application of Artificial Intelligence (AI) or machine learning (ML) techniques, the number of prediction model studies is expected to increase even further. Prediction models developed using AI or ML techniques are often labelled as a 'black box' and little is known about their methodological and reporting quality. Therefore, this comprehensive systematic review aims to evaluate the reporting quality, the methodological conduct, and the risk of bias of prediction model studies that applied ML techniques for model development and/or validation. METHODS AND ANALYSIS: A search will be performed in PubMed to identify studies developing and/or validating prediction models using any ML methodology and across all medical fields. Studies will be included if they were published between January 2018 and December 2019, predict patient-related outcomes, use any study design or data source, and available in English. Screening of search results and data extraction from included articles will be performed by two independent reviewers. The primary outcomes of this systematic review are: (1) the adherence of ML-based prediction model studies to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD), and (2) the risk of bias in such studies as assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). A narrative synthesis will be conducted for all included studies. Findings will be stratified by study type, medical field and prevalent ML methods, and will inform necessary extensions or updates of TRIPOD and PROBAST to better address prediction model studies that used AI or ML techniques. ETHICS AND DISSEMINATION: Ethical approval is not required for this study because only available published data will be analysed. Findings will be disseminated through peer-reviewed publications and scientific conferences. SYSTEMATIC REVIEW REGISTRATION: PROSPERO, CRD42019161764.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Navarro, C. L. A., Damen, J. A., Takada, T., Nijman, S. W. J., Dhiman, P., Ma, J., Collins, P. G. S., Bajpai, R. C., Riley, R. D., Moons, K. G., & Hooft, L. (2020). Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques. BMJ Open, 10(11), e038832. https://doi.org/10.1136/bmjopen-2020-038832
MLA 9
Navarro, Constanza L. Andaur, et al. “Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques.” BMJ Open, vol. 10, no. 11, 2020, p. e038832. https://doi.org/10.1136/bmjopen-2020-038832.
Chicago
Navarro, Constanza L. Andaur, Johanna AAG Damen, Toshihiko Takada, Steven W J Nijman, Paula Dhiman, Jie Ma, Professor Gary S. Collins, et al. 2020. “Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques.” BMJ Open 10 (11): e038832. https://doi.org/10.1136/bmjopen-2020-038832.
Harvard
Navarro, C. L. A. et al. (2020) ‘Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques’, BMJ Open, 10(11), p. e038832. doi:10.1136/bmjopen-2020-038832.
IEEE
C. L. A. Navarro et al., “Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques,” BMJ Open, vol. 10, no. 11, p. e038832, 2020, doi: 10.1136/bmjopen-2020-038832.
BibTeX
@article{navarro2020protocol,
title = {{Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques}},
author = {Navarro, Constanza L. Andaur and Damen, Johanna AAG and Takada, Toshihiko and Nijman, Steven W J and Dhiman, Paula and Ma, Jie and Collins, Professor Gary S. and Bajpai, Ram Chandra and Riley, Richard D and Moons, Karel G.M. and Hooft, Lotty},
journal = {BMJ Open},
year = {2020},
volume = {10},
number = {11},
pages = {e038832},
doi = {10.1136/bmjopen-2020-038832},
url = {https://doi.org/10.1136/bmjopen-2020-038832}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2021 | 12 | |
| 2022 | 31 | |
| 2023 | 11 | |
| 2024 | 10 | |
| 2025 | 18 | |
| 2026 | 6 |
Its field-weighted citation impact is 2.00: it has been cited 2.00 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- machine learning
- reporting quality
- methodological quality
- risk of bias
- tripod
- PROBAST
- diagnostic prediction model
- prognostic model
Work with this paper
Most-cited papers in Decision Sciences
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsMatthew James Page, Joanne E. McKenzie, Patrick M. Bossuyt and 23 more · BMJ · 2021 · 144,919 citations
- Evaluating Structural Equation Models with Unobservable Variables and Measurement ErrorClaes Fornell, David F. Larcker · Journal of Marketing Research · 1981 · 118,336 citations
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesLi‐tze Hu, Peter M. Bentler · Structural Equation Modeling A Multidisciplinary Journal · 1999 · 117,461 citations
- Common method biases in behavioral research: A critical review of the literature and recommended remedies.Philip M. Podsakoff, SCOTT B. MACKENZIE, Jeong Yeon Lee and 1 more · Journal of Applied Psychology · 2003 · 89,248 citations
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statementDavid Moher, Alessandro Liberati, Jennifer Tetzlaff and 2 more · BMJ · 2009 · 85,524 citations
Questions about this paper
How do I cite this paper in APA style?
Navarro, C. L. A., Damen, J. A., Takada, T., Nijman, S. W. J., Dhiman, P., Ma, J., Collins, P. G. S., Bajpai, R. C., Riley, R. D., Moons, K. G., & Hooft, L. (2020). Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques. BMJ Open, 10(11), e038832. https://doi.org/10.1136/bmjopen-2020-038832
How many times has this paper been cited?
88 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at BMJ Open.
Who wrote this paper?
Constanza L. Andaur Navarro, Johanna AAG Damen, Toshihiko Takada, Steven W J Nijman, Paula Dhiman, Jie Ma, Professor Gary S. Collins, Ram Chandra Bajpai, Richard D Riley, Karel G.M. Moons, Lotty Hooft.
What is the DOI of this paper?
10.1136/bmjopen-2020-038832. The permanent link is https://doi.org/10.1136/bmjopen-2020-038832.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.