Conference paper · 2024 · Open access
A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models
EGU General Assembly Conference Abstracts · Published 9 March 2024 · DOI 10.5194/egusphere-egu24-13853
In brief
“A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models” is a 2024 conference paper by Imme Ebert‐Uphoff and 3 co-authors, published in EGU General Assembly Conference Abstracts. OpenAlex counts 0 citations as of 15 November 2026. It is open access under a CC BY licence: the full text is free to read at EGU General Assembly Conference Abstracts.
- Citations
- 0
- Authors
- 4
Abstract
Over the past few years purely AI-driven global weather forecasting models have emerged that show increasingly impressive skill, raising the question whether AI models might soon compete with NWP models for selected forecasting tasks. At this point these AI-based models are still in the proof-of-concept stage and not ready to be used for operational forecasting, but entirely new AI-models emerge every 2-3 months, with rapidly increasing abilities. Furthermore, many of these models are orders of magnitude faster than NWP models and can run on modest computational resources enabling repeatable on-demand forecasts competitive with NWP. The low computational cost enables the creation of very large ensembles, which better represent the tails of the forecast distribution, which, if an ensemble is well calibrated, allows for better forecasting of rare and extreme events.However, these AI-based weather forecasting models have not yet been rigorously tested by the meteorological community, and their utility to operational forecasters is unknown. In this presentation we propose several studies to address the above issues, grouped into two central foci:(1) Nature of AI models: AI-based models have very different characteristics from NWP models. Thus, in addition to applying evaluation procedures developed for NWP models, we need to develop procedures that test for AI-specific weaknesses. For example, NWP models and their physics backbone guarantee certain properties - such as dynamic coupling between fields - that AI-based models are not required to uphold. Developing suitable tests is based on a fundamental understanding of the AI-based models.(2) Forecaster Perspective: Evaluation of weather forecasting models should be performed with respect to particular applications of weather forecasts, and it is critical to have research meteorologists and operational forecasters involved in the evaluation process. Our initial evaluation of AI-based models in CIRA weather briefings revealed that these models have characteristics that make interpretation of their forecasts fundamentally different from the physics-based NWP model predictions meteorologists are familiar with. For example, the increasing “blurriness” of AI-based predictions with longer lead times is not a reflection of weaker atmospheric circulations, but rather a reflection of uncertainty. Evaluations aimed at specific meteorological phenomena and atmospheric processes will allow the community to make informed decisions in the future regarding in what environments and for which applications AI-based weather forecasting models may be safe and beneficial to use.In summary, AI-based weather forecasts have different characteristics from familiar dynamically-based forecasts, and it is thus important to have a robust research plan to evaluate many different characteristics of the models in order to provide guidelines to operational forecasters and feedback to model developers.…
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Ebert‐Uphoff, I., Stewart, J. Q., Radford, J. T., & CIRA-NOAA team. (2024). A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models. EGU General Assembly Conference Abstracts. https://doi.org/10.5194/egusphere-egu24-13853
MLA 9
Ebert‐Uphoff, Imme, et al. “A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models.” EGU General Assembly Conference Abstracts, 2024. https://doi.org/10.5194/egusphere-egu24-13853.
Chicago
Ebert‐Uphoff, Imme, Jebb Q. Stewart, Jacob T. Radford, and CIRA-NOAA team. 2024. “A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models.” EGU General Assembly Conference Abstracts. https://doi.org/10.5194/egusphere-egu24-13853.
Harvard
Ebert‐Uphoff, I. et al. (2024) ‘A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models’, EGU General Assembly Conference Abstracts. doi:10.5194/egusphere-egu24-13853.
IEEE
I. Ebert‐Uphoff, J. Q. Stewart, J. T. Radford, and CIRA-NOAA team, “A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models,” EGU General Assembly Conference Abstracts, 2024, doi: 10.5194/egusphere-egu24-13853.
BibTeX
@inproceedings{ebertuphoff2024research,
title = {{A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models}},
author = {Ebert‐Uphoff, Imme and Stewart, Jebb Q. and Radford, Jacob T. and CIRA-NOAA team},
booktitle = {EGU General Assembly Conference Abstracts},
year = {2024},
doi = {10.5194/egusphere-egu24-13853},
url = {https://doi.org/10.5194/egusphere-egu24-13853}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- numerical weather prediction
- forecast evaluation
- ensemble forecasting
- forecast uncertainty
- operational forecasting
- extreme event prediction
- forecast interpretation
- forecast predictability
Work with this paper
Most-cited papers in Earth and Planetary Sciences
- The ERA5 global reanalysisHans Hersbach, Bill Bell, Paul Berrisford and 40 more · Quarterly Journal of the Royal Meteorological Society · 2020 · 35,424 citations
- The NCEP/NCAR 40-Year Reanalysis ProjectEugenia Kalnay, Masao Kanamitsu, Robert E. Kistler and 19 more · Bulletin of the American Meteorological Society · 1996 · 28,486 citations
- Chemical and isotopic systematics of oceanic basalts: implications for mantle composition and processesShen‐Su Sun, W. F. McDonough · Geological Society London Special Publications · 1989 · 26,603 citations
- The ERA‐Interim reanalysis: configuration and performance of the data assimilation systemDick P. Dee, Sakari M. Uppala, A. J. Simmons and 33 more · Quarterly Journal of the Royal Meteorological Society · 2011 · 25,245 citations
- PAST: PALEONTOLOGICAL STATISTICAL SOFTWARE PACKAGE FOR EDUCATION AND DATA ANALYSISØyvind Hammer, David A. T. Harper, Paul D. Ryan · 2001 · 17,871 citations
Questions about this paper
How do I cite this paper in APA style?
Ebert‐Uphoff, I., Stewart, J. Q., Radford, J. T., & CIRA-NOAA team. (2024). A Research Agenda for the Evaluation of AI-Based Weather Forecasting Models. EGU General Assembly Conference Abstracts. https://doi.org/10.5194/egusphere-egu24-13853
How many times has this paper been cited?
0 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at EGU General Assembly Conference Abstracts.
Who wrote this paper?
Imme Ebert‐Uphoff, Jebb Q. Stewart, Jacob T. Radford, CIRA-NOAA team.
What is the DOI of this paper?
10.5194/egusphere-egu24-13853. The permanent link is https://doi.org/10.5194/egusphere-egu24-13853.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.