Conference paper · 2025 · Open access

On the robustness of AI model forecast skill and initial condition uncertainty of the 2021 Pacific Northwest Heatwave

Osamu Miyawaki, Cuiyi Fei, Siyu Li, Dhruvit Patel, Giorgio Sarro, Huan Zhang, Adam Marchakitus, Pedram Hassanzadeh, Dorian S. Abbot, Jonathan Weare, Noboru Nakamura, Tiffany A. Shaw

EGU General Assembly Conference Abstracts · Published 14 March 2025 · DOI 10.5194/egusphere-egu25-4576

Open full textCite this paper

In brief

“On the robustness of AI model forecast skill and initial condition uncertainty of the 2021 Pacific Northwest Heatwave” is a 2025 conference paper by Osamu Miyawaki and 11 co-authors, published in EGU General Assembly Conference Abstracts. OpenAlex counts 0 citations as of 20 November 2026. It is open access under a CC BY licence: the full text is free to read at EGU General Assembly Conference Abstracts.

Citations
0
Authors
12

Abstract

AI weather models are becoming valuable tools for predicting the weather. While AI models’ general forecasts are known to be skillful, their forecast skill of extreme events is not fully understood. The 2021 Pacific Northwest (PNW) heatwave is a good case study for AI models because it falls outside of the distribution of heat waves in AI model training datasets.Here, we investigate the forecast performance of 8 AI models (AIFS, Gencast, NeuralGCM, Graphcast, Fuxi, Pangu, Fourcastnet, FourcastnetV2) of the 2021 PNW heatwave. Despite the event being out of the training dataset distribution, their forecast performance is comparable to that of a state-of-the-art numerical weather prediction model (IFS). Specifically, AI models and IFS can accurately forecast the heatwave for lead times less than 7 days.Two recent studies suggest the predictability barrier of the PNW heatwave may be due to an initial condition observation error. Leach et al. (2024) found that the 26th ensemble member of a 250 member IFS forecast accurately forecasts the heatwave 12 days in advance. Vonich and Hakim (2024) used backpropagation in Graphcast to find an optimal initial condition that leads to an accurate forecast 10 days in advance. Are these initial conditions robust across an ensemble of AI models? And do these initial conditions point to a unique solution?We find a large spread in forecast accuracy when running the 8 AI models with the Leach et al. (2024) and Vonich & Hakim (2024) initial conditions. Furthermore we ran 1000 member ensembles in NeuralGCM and find initial conditions that lead to an accurate long-term forecast are not unique. These results suggest that the improvement in forecast accuracty to certain initial conditions may not necessarily be due to the initial conditions being closer to ground truth but rather they are due to cancelation of model error.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Miyawaki, O., Fei, C., Li, S., Patel, D., Sarro, G., Zhang, H., Marchakitus, A., Hassanzadeh, P., Abbot, D. S., Weare, J., Nakamura, N., & Shaw, T. A. (2025). On the robustness of AI model forecast skill and initial condition uncertainty of the 2021 Pacific Northwest Heatwave. EGU General Assembly Conference Abstracts. https://doi.org/10.5194/egusphere-egu25-4576

MLA 9

Miyawaki, Osamu, et al. “On the robustness of AI model forecast skill and initial condition uncertainty of the 2021 Pacific Northwest Heatwave.” EGU General Assembly Conference Abstracts, 2025. https://doi.org/10.5194/egusphere-egu25-4576.

Chicago

Miyawaki, Osamu, Cuiyi Fei, Siyu Li, Dhruvit Patel, Giorgio Sarro, Huan Zhang, Adam Marchakitus, et al. 2025. “On the robustness of AI model forecast skill and initial condition uncertainty of the 2021 Pacific Northwest Heatwave.” EGU General Assembly Conference Abstracts. https://doi.org/10.5194/egusphere-egu25-4576.

Harvard

Miyawaki, O. et al. (2025) ‘On the robustness of AI model forecast skill and initial condition uncertainty of the 2021 Pacific Northwest Heatwave’, EGU General Assembly Conference Abstracts. doi:10.5194/egusphere-egu25-4576.

IEEE

O. Miyawaki et al., “On the robustness of AI model forecast skill and initial condition uncertainty of the 2021 Pacific Northwest Heatwave,” EGU General Assembly Conference Abstracts, 2025, doi: 10.5194/egusphere-egu25-4576.

BibTeX

@inproceedings{miyawaki2025robustness,
  title     = {{On the robustness of AI model forecast skill and initial condition uncertainty of the 2021 Pacific Northwest Heatwave}},
  author    = {Miyawaki, Osamu and Fei, Cuiyi and Li, Siyu and Patel, Dhruvit and Sarro, Giorgio and Zhang, Huan and Marchakitus, Adam and Hassanzadeh, Pedram and Abbot, Dorian S. and Weare, Jonathan and Nakamura, Noboru and Shaw, Tiffany A.},
  booktitle = {EGU General Assembly Conference Abstracts},
  year      = {2025},
  doi       = {10.5194/egusphere-egu25-4576},
  url       = {https://doi.org/10.5194/egusphere-egu25-4576}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations

  • 2021 Pacific Northwest heatwave
  • forecast skill
  • initial condition uncertainty
  • predictability barrier
  • ensemble forecasting
  • GraphCast
  • FourCastNet
  • numerical weather prediction comparison

Work with this paper

Most-cited papers in Earth and Planetary Sciences

All highly cited Earth and Planetary Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Miyawaki, O., Fei, C., Li, S., Patel, D., Sarro, G., Zhang, H., Marchakitus, A., Hassanzadeh, P., Abbot, D. S., Weare, J., Nakamura, N., & Shaw, T. A. (2025). On the robustness of AI model forecast skill and initial condition uncertainty of the 2021 Pacific Northwest Heatwave. EGU General Assembly Conference Abstracts. https://doi.org/10.5194/egusphere-egu25-4576

How many times has this paper been cited?

0 times, according to OpenAlex on 20 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at EGU General Assembly Conference Abstracts: sign in and open the full text from this page.

Who wrote this paper?

Osamu Miyawaki, Cuiyi Fei, Siyu Li, Dhruvit Patel, Giorgio Sarro, Huan Zhang, Adam Marchakitus, Pedram Hassanzadeh, Dorian S. Abbot, Jonathan Weare, Noboru Nakamura, Tiffany A. Shaw.

What is the DOI of this paper?

10.5194/egusphere-egu25-4576, permanently at https://doi.org/10.5194/egusphere-egu25-4576. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 20 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.