Dissertation · 2023 · Open access

QUALITY OF MACHINE LEARNING PREDICTION MODELS IN HEALTHCARE

Constanza L. Andaur Navarro

Utrecht University Repository (Utrecht University) · Published 18 April 2023 · DOI 10.33540/1394

Free PDFView at the publisherCite this paper

In brief

“QUALITY OF MACHINE LEARNING PREDICTION MODELS IN HEALTHCARE” is a 2023 dissertation by Constanza L. Andaur Navarro, published in Utrecht University Repository (Utrecht University). OpenAlex counts 1 citations as of 15 November 2026. It is open access: the full text is free to read at Utrecht University Repository (Utrecht University).

Citations
1
Most cited in
2024 (1)
Authors
1

Abstract

For the implementation of valuable prediction models in clinical practice, properly conducted and well reported studies on early stages of model development and validation are essential. We systematically reviewed the adherence of 152 studies on machine learning-based prediction models to the 22-item checklist with the minimum standards for high quality reporting, TRIPOD. Overall, articles adhered to a median of 38.7% of applicable TRIPOD items. We identified that TRIPOD requires new items to cover AI-related aspects. We systematically reviewed the included studies for 15 spin practices and 11 poor reporting standards. A considerable number of studies lack a pre-specified protocol, make claims of clinical applicability (without further validation), and limitations are neither reported nor discussed in the context of previously developed models. Given that a first approach to spin evaluation using a classification scheme for studies on prognostic factors proved to be inefficient, we present SPIN-PM, a new framework for spin identification tailored to studies on prediction models. We provide a detailed overview of the study design, modelling strategies, and performance measures reported in studies on machine learning-based prediction models. Most studies reported only the development of prediction models and focused on binary outcomes. Within the 152 studies, we evaluated 522 models (on average 9.4 models per study), in which the most common modelling algorithms used were support vector machine and random forest. We comprehensively reviewed the methodological quality and risk of bias of studies on prediction models developed using machine learning techniques. We applied PROBAST to 152 studies on model development and 19 external validations. Of these 171 analyses, 148 were rated at high risk of bias. Efforts to improve the design, conduct, reporting, and validation of such studies are necessary to boost the introduction of machine learning based prediction models into clinical practice. We compared the absolute risk probabilities of three different modelling techniques: logistic regression, random forest, and support vector machine. For the last two techniques, we applied two different implementation methods within the package ‘caret’ in the statistical software R. Using logistic regression as benchmark, we showed that risk probabilities for deep venous thrombosis vary substantially between modelling techniques and implementation methods. TRIPOD and PROBAST were published to facilitate the critical appraisal of studies on diagnostic and prognostic prediction models. We described the five stages for the development of both extensions to machine learning-based models. The systematic reviews presented in this thesis compromised stage one. A survey using the Delphi methodology constitute stage 2 and the results are briefly discussed in Chapter 12. In conclusion, we have…

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Navarro, C. L. A. (2023). QUALITY OF MACHINE LEARNING PREDICTION MODELS IN HEALTHCARE. Utrecht University Repository (Utrecht University). https://doi.org/10.33540/1394

MLA 9

Navarro, Constanza L. Andaur. “QUALITY OF MACHINE LEARNING PREDICTION MODELS IN HEALTHCARE.” Utrecht University Repository (Utrecht University), 2023. https://doi.org/10.33540/1394.

Chicago

Navarro, Constanza L. Andaur. 2023. “QUALITY OF MACHINE LEARNING PREDICTION MODELS IN HEALTHCARE.” Utrecht University Repository (Utrecht University). https://doi.org/10.33540/1394.

Harvard

Navarro, C. L. A. (2023) ‘QUALITY OF MACHINE LEARNING PREDICTION MODELS IN HEALTHCARE’, Utrecht University Repository (Utrecht University). doi:10.33540/1394.

IEEE

C. L. A. Navarro, “QUALITY OF MACHINE LEARNING PREDICTION MODELS IN HEALTHCARE,” Utrecht University Repository (Utrecht University), 2023, doi: 10.33540/1394.

BibTeX

@phdthesis{navarro2023quality,
  title     = {{QUALITY OF MACHINE LEARNING PREDICTION MODELS IN HEALTHCARE}},
  author    = {Navarro, Constanza L. Andaur},
  year      = {2023},
  doi       = {10.33540/1394},
  url       = {https://doi.org/10.33540/1394}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews

  • machine learning prediction models
  • TRIPOD checklist
  • reporting quality
  • PROBAST
  • risk of bias
  • external validation
  • deep vein thrombosis
  • logistic regression

Work with this paper

Most-cited papers in Decision Sciences

All highly cited Decision Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Navarro, C. L. A. (2023). QUALITY OF MACHINE LEARNING PREDICTION MODELS IN HEALTHCARE. Utrecht University Repository (Utrecht University). https://doi.org/10.33540/1394

How many times has this paper been cited?

1 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at Utrecht University Repository (Utrecht University).

Who wrote this paper?

Constanza L. Andaur Navarro.

What is the DOI of this paper?

10.33540/1394. The permanent link is https://doi.org/10.33540/1394.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.