Journal article · 2024 · Open access
Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events
Artificial Intelligence for the Earth Systems, 4(1) · Published 3 December 2024 · DOI 10.1175/aies-d-24-0033.1
In brief
“Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events” is a 2024 journal article by Olivier C. Pasche and 4 co-authors, published in Artificial Intelligence for the Earth Systems. OpenAlex counts 54 citations as of 15 November 2026, which puts it in the top 7% of works published in 2024 by citations. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 54
- Most cited in
- 2026 (32)
- Field-weighted impact
- 7.88
- References
- 65
- Authors
- 5
Abstract
The forecast accuracy of machine learning (ML) weather prediction models is improving rapidly, leading many to speak of a “second revolution in weather forecasting.” With numerous methods being developed and limited physical guarantees offered by ML models, there is a critical need for a comprehensive evaluation of these emerging techniques. While this need has been partly fulfilled by benchmark datasets, they provide little information on rare and impactful extreme events or on compound impact metrics, for which model accuracy might degrade due to misrepresented dependencies between variables. To address these issues, we compare ML weather prediction models (GraphCast, Pangu-Weather, and FourCastNet) and ECMWF’s high-resolution forecast system (HRES) in three case studies: the 2021 Pacific Northwest heatwave, the 2023 South Asian humid heatwave, and the North American winter storm in 2021. We find that ML weather prediction models locally achieve similar accuracy to HRES on the record-shattering Pacific Northwest heatwave but underperform when aggregated over space and time. However, they forecast the compound winter storm substantially better. We also highlight structural differences in how the errors of HRES and the ML models build up to that event. The ML forecasts lack important variables for a detailed assessment of the health risks of the 2023 humid heatwave. Using a possible substitute variable, prediction errors show spatial patterns with the highest danger levels over Bangladesh being underestimated by the ML models. Generally, case study–driven, impact-centric evaluation can complement existing research, increase public trust, and aid in developing reliable ML weather prediction models. Significance Statement With the performance of machine learning–based weather forecasting models improving rapidly, thorough analyses are needed to ensure that their forecasts are accurate and reliable before deploying them in operational settings. Existing evaluations often reduce forecast performance to a few metrics, potentially obscuring rare but systematic errors. This is especially problematic for high-impact extreme events, which, by definition, are rare in the data but often substantially affect society. In a detailed analysis of three extreme events, we observe that, although machine learning (ML) models generally outperform the best physics-based numerical weather prediction (NWP) model on benchmark datasets, they do not consistently do so for the studied extreme events or compound impact metrics and lack some impact-relevant variables.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Pasche, O. C., Wider, J., Zhang, Z., A, J. Z., & Engelke, S. (2024). Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events. Artificial Intelligence for the Earth Systems, 4(1). https://doi.org/10.1175/aies-d-24-0033.1
MLA 9
Pasche, Olivier C., et al. “Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events.” Artificial Intelligence for the Earth Systems, vol. 4, no. 1, 2024. https://doi.org/10.1175/aies-d-24-0033.1.
Chicago
Pasche, Olivier C., Jonathan Wider, Zhongwei Zhang, Jakob Zscheischler A, and Sebastian Engelke. 2024. “Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events.” Artificial Intelligence for the Earth Systems 4 (1). https://doi.org/10.1175/aies-d-24-0033.1.
Harvard
Pasche, O. C. et al. (2024) ‘Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events’, Artificial Intelligence for the Earth Systems, 4(1). doi:10.1175/aies-d-24-0033.1.
IEEE
O. C. Pasche, J. Wider, Z. Zhang, J. Z. A, and S. Engelke, “Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events,” Artificial Intelligence for the Earth Systems, vol. 4, no. 1, 2024, doi: 10.1175/aies-d-24-0033.1.
BibTeX
@article{pasche2024validating,
title = {{Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events}},
author = {Pasche, Olivier C. and Wider, Jonathan and Zhang, Zhongwei and A, Jakob Zscheischler and Engelke, Sebastian},
journal = {Artificial Intelligence for the Earth Systems},
year = {2024},
volume = {4},
number = {1},
doi = {10.1175/aies-d-24-0033.1},
url = {https://doi.org/10.1175/aies-d-24-0033.1}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2024 | 2 | |
| 2025 | 20 | |
| 2026 | 32 |
Its field-weighted citation impact is 7.88: it has been cited 7.88 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- extreme weather events
- forecast evaluation
- heat waves
- winter storms
- GraphCast
- Pangu-Weather
- FourCastNet
- 2021 Pacific Northwest heatwave
Work with this paper
Most-cited papers in Earth and Planetary Sciences
- The ERA5 global reanalysisHans Hersbach, Bill Bell, Paul Berrisford and 40 more · Quarterly Journal of the Royal Meteorological Society · 2020 · 35,424 citations
- The NCEP/NCAR 40-Year Reanalysis ProjectEugenia Kalnay, Masao Kanamitsu, Robert E. Kistler and 19 more · Bulletin of the American Meteorological Society · 1996 · 28,486 citations
- Chemical and isotopic systematics of oceanic basalts: implications for mantle composition and processesShen‐Su Sun, W. F. McDonough · Geological Society London Special Publications · 1989 · 26,603 citations
- The ERA‐Interim reanalysis: configuration and performance of the data assimilation systemDick P. Dee, Sakari M. Uppala, A. J. Simmons and 33 more · Quarterly Journal of the Royal Meteorological Society · 2011 · 25,245 citations
- PAST: PALEONTOLOGICAL STATISTICAL SOFTWARE PACKAGE FOR EDUCATION AND DATA ANALYSISØyvind Hammer, David A. T. Harper, Paul D. Ryan · 2001 · 17,871 citations
Questions about this paper
How do I cite this paper in APA style?
Pasche, O. C., Wider, J., Zhang, Z., A, J. Z., & Engelke, S. (2024). Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events. Artificial Intelligence for the Earth Systems, 4(1). https://doi.org/10.1175/aies-d-24-0033.1
How many times has this paper been cited?
54 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at arXiv (Cornell University).
Who wrote this paper?
Olivier C. Pasche, Jonathan Wider, Zhongwei Zhang, Jakob Zscheischler A, Sebastian Engelke.
What is the DOI of this paper?
10.1175/aies-d-24-0033.1. The permanent link is https://doi.org/10.1175/aies-d-24-0033.1.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.