Journal article · 2022 · Open access
Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database
PLOS Digital Health, 1(4), e0000023 · Published 5 April 2022 · DOI 10.1371/journal.pdig.0000023
In brief
“Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database” is a 2022 journal article by Harvineet Singh and 2 co-authors, published in PLOS Digital Health. OpenAlex counts 76 citations as of 16 November 2026, which puts it in the top 3% of works published in 2022 by citations. It is open access under a CC BY licence: the full text is free to read at PLOS Digital Health.
- Citations
- 76
- Most cited in
- 2026 (24)
- Field-weighted impact
- 3.43
- References
- 91
- Authors
- 3
Abstract
Modern predictive models require large amounts of data for training and evaluation, absence of which may result in models that are specific to certain locations, populations in them and clinical practices. Yet, best practices for clinical risk prediction models have not yet considered such challenges to generalizability. Here we ask whether population- and group-level performance of mortality prediction models vary significantly when applied to hospitals or geographies different from the ones in which they are developed. Further, what characteristics of the datasets explain the performance variation? In this multi-center cross-sectional study, we analyzed electronic health records from 179 hospitals across the US with 70,126 hospitalizations from 2014 to 2015. Generalization gap, defined as difference between model performance metrics across hospitals, is computed for area under the receiver operating characteristic curve (AUC) and calibration slope. To assess model performance by the race variable, we report differences in false negative rates across groups. Data were also analyzed using a causal discovery algorithm “Fast Causal Inference” that infers paths of causal influence while identifying potential influences associated with unmeasured variables. When transferring models across hospitals, AUC at the test hospital ranged from 0.777 to 0.832 (1st-3rd quartile or IQR; median 0.801); calibration slope from 0.725 to 0.983 (IQR; median 0.853); and disparity in false negative rates from 0.046 to 0.168 (IQR; median 0.092). Distribution of all variable types (demography, vitals, and labs) differed significantly across hospitals and regions. The race variable also mediated differences in the relationship between clinical variables and mortality, by hospital/region. In conclusion, group-level performance should be assessed during generalizability checks to identify potential harms to the groups. Moreover, for developing methods to improve model performance in new environments, a better understanding and documentation of provenance of data and health processes are needed to identify and mitigate sources of variation.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Singh, H., Mhasawade, V., & Chunara, R. (2022). Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database. PLOS Digital Health, 1(4), e0000023. https://doi.org/10.1371/journal.pdig.0000023
MLA 9
Singh, Harvineet, et al. “Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database.” PLOS Digital Health, vol. 1, no. 4, 2022, p. e0000023. https://doi.org/10.1371/journal.pdig.0000023.
Chicago
Singh, Harvineet, Vishwali Mhasawade, and Rumi Chunara. 2022. “Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database.” PLOS Digital Health 1 (4): e0000023. https://doi.org/10.1371/journal.pdig.0000023.
Harvard
Singh, H., Mhasawade, V. and Chunara, R. (2022) ‘Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database’, PLOS Digital Health, 1(4), p. e0000023. doi:10.1371/journal.pdig.0000023.
IEEE
H. Singh, V. Mhasawade, and R. Chunara, “Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database,” PLOS Digital Health, vol. 1, no. 4, p. e0000023, 2022, doi: 10.1371/journal.pdig.0000023.
BibTeX
@article{singh2022generalizability,
title = {{Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database}},
author = {Singh, Harvineet and Mhasawade, Vishwali and Chunara, Rumi},
journal = {PLOS Digital Health},
year = {2022},
volume = {1},
number = {4},
pages = {e0000023},
doi = {10.1371/journal.pdig.0000023},
url = {https://doi.org/10.1371/journal.pdig.0000023}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2022 | 5 | |
| 2023 | 12 | |
| 2024 | 16 | |
| 2025 | 19 | |
| 2026 | 24 |
Its field-weighted citation impact is 3.43: it has been cited 3.43 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- mortality risk prediction model
- model generalization
- generalization gap
- racial disparities
- false negative rate
- calibration slope
- electronic health records
- causal discovery
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Singh, H., Mhasawade, V., & Chunara, R. (2022). Generalizability challenges of mortality risk prediction models: A retrospective analysis on a multi-center database. PLOS Digital Health, 1(4), e0000023. https://doi.org/10.1371/journal.pdig.0000023
How many times has this paper been cited?
76 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at PLOS Digital Health.
Who wrote this paper?
Harvineet Singh, Vishwali Mhasawade, Rumi Chunara.
What is the DOI of this paper?
10.1371/journal.pdig.0000023. The permanent link is https://doi.org/10.1371/journal.pdig.0000023.
Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.