Journal article · 2024 · Open access
Multimodal risk prediction with physiological signals, medical images and clinical notes
Heliyon, 10(5), e26772 · Published 28 February 2024 · DOI 10.1016/j.heliyon.2024.e26772
In brief
“Multimodal risk prediction with physiological signals, medical images and clinical notes” is a 2024 journal article by Yuanlong Wang and 2 co-authors, published in Heliyon. OpenAlex counts 97 citations as of 19 November 2026, which puts it in the top 2% of works published in 2024 by citations. It is open access under a CC BY-NC-ND licence: the full text is free to read at Heliyon.
- Citations
- 97
- Most cited in
- 2026 (52)
- Field-weighted impact
- 7.39
- References
- 35
- Authors
- 3
Abstract
The broad adoption of electronic health record (EHR) systems brings us a tremendous amount of clinical data and thus provides opportunities to conduct data-based healthcare research to solve various clinical problems in the medical domain. Machine learning and deep learning methods are widely used in the medical informatics and healthcare domain due to their power to mine insights from raw data. When adapting deep learning models for EHR data, it is essential to consider its heterogeneous nature: EHR contains patient records from various sources including medical tests (e.g. blood test, microbiology test), medical imaging, diagnosis, medications, procedures, clinical notes, etc. Those modalities together provide a holistic view of patient health status and complement each other. Therefore, combining data from multiple modalities that are intrinsically different is challenging but intuitively promising in deep learning for EHR. To assess the expectations of multimodal data, we introduce a comprehensive fusion framework designed to integrate temporal variables, medical images, and clinical notes in EHR for enhanced performance in clinical risk prediction. Early, joint, and late fusion strategies are employed to combine data from various modalities effectively. We test the model with three predictive tasks: in-hospital mortality, long length of stay, and 30-day readmission. Experimental results show that multimodal models outperform uni-modal models in the tasks involved. Additionally, by training models with different input modality combinations, we calculate the Shapley value for each modality to quantify their contribution to multimodal performance. It is shown that temporal variables tend to be more helpful than CXR images and clinical notes in the three explored predictive tasks.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Wang, Y., Yin, C., & Zhang, P. (2024). Multimodal risk prediction with physiological signals, medical images and clinical notes. Heliyon, 10(5), e26772. https://doi.org/10.1016/j.heliyon.2024.e26772
MLA 9
Wang, Yuanlong, et al. “Multimodal risk prediction with physiological signals, medical images and clinical notes.” Heliyon, vol. 10, no. 5, 2024, p. e26772. https://doi.org/10.1016/j.heliyon.2024.e26772.
Chicago
Wang, Yuanlong, Changchang Yin, and Ping Zhang. 2024. “Multimodal risk prediction with physiological signals, medical images and clinical notes.” Heliyon 10 (5): e26772. https://doi.org/10.1016/j.heliyon.2024.e26772.
Harvard
Wang, Y., Yin, C. and Zhang, P. (2024) ‘Multimodal risk prediction with physiological signals, medical images and clinical notes’, Heliyon, 10(5), p. e26772. doi:10.1016/j.heliyon.2024.e26772.
IEEE
Y. Wang, C. Yin, and P. Zhang, “Multimodal risk prediction with physiological signals, medical images and clinical notes,” Heliyon, vol. 10, no. 5, p. e26772, 2024, doi: 10.1016/j.heliyon.2024.e26772.
BibTeX
@article{wang2024multimodal,
title = {{Multimodal risk prediction with physiological signals, medical images and clinical notes}},
author = {Wang, Yuanlong and Yin, Changchang and Zhang, Ping},
journal = {Heliyon},
year = {2024},
volume = {10},
number = {5},
pages = {e26772},
doi = {10.1016/j.heliyon.2024.e26772},
url = {https://doi.org/10.1016/j.heliyon.2024.e26772}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2024 | 9 | |
| 2025 | 36 | |
| 2026 | 52 |
Its field-weighted citation impact is 7.39: it has been cited 7.39 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- multimodal fusion
- electronic health records
- clinical risk prediction
- in-hospital mortality
- length of hospital stay
- 30-day readmission
- clinical notes
- chest X-ray images
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Wang, Y., Yin, C., & Zhang, P. (2024). Multimodal risk prediction with physiological signals, medical images and clinical notes. Heliyon, 10(5), e26772. https://doi.org/10.1016/j.heliyon.2024.e26772
How many times has this paper been cited?
97 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Heliyon: sign in and open the full text from this page.
Who wrote this paper?
Yuanlong Wang, Changchang Yin, Ping Zhang.
What is the DOI of this paper?
10.1016/j.heliyon.2024.e26772, permanently at https://doi.org/10.1016/j.heliyon.2024.e26772. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.