Journal article · 2022 · Open access

Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database

Harvineet Singh, Vishwali Mhasawade, Rumi Chunara

PLOS Digital Health, 1(4), e0000023 · Published 5 April 2022 · DOI 10.1371/journal.pdig.0000023

Read the free full textCite this paper

In brief

“Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database” is a 2022 journal article by Harvineet Singh and 2 co-authors, published in PLOS Digital Health. OpenAlex counts 76 citations as of 16 November 2026, which puts it in the top 3% of works published in 2022 by citations. It is open access under a CC BY licence: the full text is free to read at PLOS Digital Health.

Citations
76
Most cited in
2026 (24)
Field-weighted impact
3.43
References
91
Authors
3

Abstract

Modern predictive models require large amounts of data for training and evaluation, absence of which may result in models that are specific to certain locations, populations in them and clinical practices. Yet, best practices for clinical risk prediction models have not yet considered such challenges to generalizability. Here we ask whether population- and group-level performance of mortality prediction models vary significantly when applied to hospitals or geographies different from the ones in which they are developed. Further, what characteristics of the datasets explain the performance variation? In this multi-center cross-sectional study, we analyzed electronic health records from 179 hospitals across the US with 70,126 hospitalizations from 2014 to 2015. Generalization gap, defined as difference between model performance metrics across hospitals, is computed for area under the receiver operating characteristic curve (AUC) and calibration slope. To assess model performance by the race variable, we report differences in false negative rates across groups. Data were also analyzed using a causal discovery algorithm “Fast Causal Inference” that infers paths of causal influence while identifying potential influences associated with unmeasured variables. When transferring models across hospitals, AUC at the test hospital ranged from 0.777 to 0.832 (1st-3rd quartile or IQR; median 0.801); calibration slope from 0.725 to 0.983 (IQR; median 0.853); and disparity in false negative rates from 0.046 to 0.168 (IQR; median 0.092). Distribution of all variable types (demography, vitals, and labs) differed significantly across hospitals and regions. The race variable also mediated differences in the relationship between clinical variables and mortality, by hospital/region. In conclusion, group-level performance should be assessed during generalizability checks to identify potential harms to the groups. Moreover, for developing methods to improve model performance in new environments, a better understanding and documentation of provenance of data and health processes are needed to identify and mitigate sources of variation.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Singh, H., Mhasawade, V., & Chunara, R. (2022). Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database. PLOS Digital Health, 1(4), e0000023. https://doi.org/10.1371/journal.pdig.0000023

MLA 9

Singh, Harvineet, et al. “Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database.” PLOS Digital Health, vol. 1, no. 4, 2022, p. e0000023. https://doi.org/10.1371/journal.pdig.0000023.

Chicago

Singh, Harvineet, Vishwali Mhasawade, and Rumi Chunara. 2022. “Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database.” PLOS Digital Health 1 (4): e0000023. https://doi.org/10.1371/journal.pdig.0000023.

Harvard

Singh, H., Mhasawade, V. and Chunara, R. (2022) ‘Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database’, PLOS Digital Health, 1(4), p. e0000023. doi:10.1371/journal.pdig.0000023.

IEEE

H. Singh, V. Mhasawade, and R. Chunara, “Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database,” PLOS Digital Health, vol. 1, no. 4, p. e0000023, 2022, doi: 10.1371/journal.pdig.0000023.

BibTeX

@article{singh2022generalizability,
  title     = {{Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database}},
  author    = {Singh, Harvineet and Mhasawade, Vishwali and Chunara, Rumi},
  journal   = {PLOS Digital Health},
  year      = {2022},
  volume    = {1},
  number    = {4},
  pages     = {e0000023},
  doi       = {10.1371/journal.pdig.0000023},
  url       = {https://doi.org/10.1371/journal.pdig.0000023}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202255
20231212
20241616
20251919
20262424

Its field-weighted citation impact is 3.43: it has been cited 3.43 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare

  • mortality risk prediction model
  • model generalization
  • generalization gap
  • racial disparities
  • false negative rate
  • calibration slope
  • electronic health records
  • causal discovery

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Singh, H., Mhasawade, V., & Chunara, R. (2022). Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database. PLOS Digital Health, 1(4), e0000023. https://doi.org/10.1371/journal.pdig.0000023

How many times has this paper been cited?

76 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at PLOS Digital Health.

Who wrote this paper?

Harvineet Singh, Vishwali Mhasawade, Rumi Chunara.

What is the DOI of this paper?

10.1371/journal.pdig.0000023. The permanent link is https://doi.org/10.1371/journal.pdig.0000023.

Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.