Journal article · 2024 · Open access

Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events

Olivier C. Pasche, Jonathan Wider, Zhongwei Zhang, Jakob Zscheischler A, Sebastian Engelke

Artificial Intelligence for the Earth Systems, 4(1) · Published 3 December 2024 · DOI 10.1175/aies-d-24-0033.1

Free PDFView at the publisherCite this paper

In brief

“Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events” is a 2024 journal article by Olivier C. Pasche and 4 co-authors, published in Artificial Intelligence for the Earth Systems. OpenAlex counts 54 citations as of 15 November 2026, which puts it in the top 7% of works published in 2024 by citations. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).

Citations
54
Most cited in
2026 (32)
Field-weighted impact
7.88
References
65
Authors
5

Abstract

The forecast accuracy of machine learning (ML) weather prediction models is improving rapidly, leading many to speak of a “second revolution in weather forecasting.” With numerous methods being developed and limited physical guarantees offered by ML models, there is a critical need for a comprehensive evaluation of these emerging techniques. While this need has been partly fulfilled by benchmark datasets, they provide little information on rare and impactful extreme events or on compound impact metrics, for which model accuracy might degrade due to misrepresented dependencies between variables. To address these issues, we compare ML weather prediction models (GraphCast, Pangu-Weather, and FourCastNet) and ECMWF’s high-resolution forecast system (HRES) in three case studies: the 2021 Pacific Northwest heatwave, the 2023 South Asian humid heatwave, and the North American winter storm in 2021. We find that ML weather prediction models locally achieve similar accuracy to HRES on the record-shattering Pacific Northwest heatwave but underperform when aggregated over space and time. However, they forecast the compound winter storm substantially better. We also highlight structural differences in how the errors of HRES and the ML models build up to that event. The ML forecasts lack important variables for a detailed assessment of the health risks of the 2023 humid heatwave. Using a possible substitute variable, prediction errors show spatial patterns with the highest danger levels over Bangladesh being underestimated by the ML models. Generally, case study–driven, impact-centric evaluation can complement existing research, increase public trust, and aid in developing reliable ML weather prediction models. Significance Statement With the performance of machine learning–based weather forecasting models improving rapidly, thorough analyses are needed to ensure that their forecasts are accurate and reliable before deploying them in operational settings. Existing evaluations often reduce forecast performance to a few metrics, potentially obscuring rare but systematic errors. This is especially problematic for high-impact extreme events, which, by definition, are rare in the data but often substantially affect society. In a detailed analysis of three extreme events, we observe that, although machine learning (ML) models generally outperform the best physics-based numerical weather prediction (NWP) model on benchmark datasets, they do not consistently do so for the studied extreme events or compound impact metrics and lack some impact-relevant variables.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Pasche, O. C., Wider, J., Zhang, Z., A, J. Z., & Engelke, S. (2024). Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events. Artificial Intelligence for the Earth Systems, 4(1). https://doi.org/10.1175/aies-d-24-0033.1

MLA 9

Pasche, Olivier C., et al. “Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events.” Artificial Intelligence for the Earth Systems, vol. 4, no. 1, 2024. https://doi.org/10.1175/aies-d-24-0033.1.

Chicago

Pasche, Olivier C., Jonathan Wider, Zhongwei Zhang, Jakob Zscheischler A, and Sebastian Engelke. 2024. “Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events.” Artificial Intelligence for the Earth Systems 4 (1). https://doi.org/10.1175/aies-d-24-0033.1.

Harvard

Pasche, O. C. et al. (2024) ‘Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events’, Artificial Intelligence for the Earth Systems, 4(1). doi:10.1175/aies-d-24-0033.1.

IEEE

O. C. Pasche, J. Wider, Z. Zhang, J. Z. A, and S. Engelke, “Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events,” Artificial Intelligence for the Earth Systems, vol. 4, no. 1, 2024, doi: 10.1175/aies-d-24-0033.1.

BibTeX

@article{pasche2024validating,
  title     = {{Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events}},
  author    = {Pasche, Olivier C. and Wider, Jonathan and Zhang, Zhongwei and A, Jakob Zscheischler and Engelke, Sebastian},
  journal   = {Artificial Intelligence for the Earth Systems},
  year      = {2024},
  volume    = {4},
  number    = {1},
  doi       = {10.1175/aies-d-24-0033.1},
  url       = {https://doi.org/10.1175/aies-d-24-0033.1}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202422
20252020
20263232

Its field-weighted citation impact is 7.88: it has been cited 7.88 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations

  • extreme weather events
  • forecast evaluation
  • heat waves
  • winter storms
  • GraphCast
  • Pangu-Weather
  • FourCastNet
  • 2021 Pacific Northwest heatwave

Work with this paper

Most-cited papers in Earth and Planetary Sciences

All highly cited Earth and Planetary Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Pasche, O. C., Wider, J., Zhang, Z., A, J. Z., & Engelke, S. (2024). Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events. Artificial Intelligence for the Earth Systems, 4(1). https://doi.org/10.1175/aies-d-24-0033.1

How many times has this paper been cited?

54 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at arXiv (Cornell University).

Who wrote this paper?

Olivier C. Pasche, Jonathan Wider, Zhongwei Zhang, Jakob Zscheischler A, Sebastian Engelke.

What is the DOI of this paper?

10.1175/aies-d-24-0033.1. The permanent link is https://doi.org/10.1175/aies-d-24-0033.1.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.