Preprint · 2026 · Open access
Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS
arXiv (Cornell University) · Published 19 June 2026 · DOI 10.48550/arxiv.2606.21170
In brief
“Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS” is a 2026 preprint by Tobias Biegert and 3 co-authors, published in arXiv (Cornell University). OpenAlex counts 3 citations as of 15 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 3
- Most cited in
- 2026 (3)
- References
- 6
- Authors
- 4
Abstract
We study whether deterministic AI weather prediction (AIWP) models issue more informative forecasts for extreme weather events than deterministic numerical weather prediction (NWP) models. The deterministic model output is subjected to statistical post-processing via isotonic distributional regression (IDR), or EasyUQ, before the resulting probabilistic forecasts are assessed using weighted versions of the continuous ranked probability score (CRPS). This extends the Potential CRPS (PCRPS) measure proposed by Gneiting et al. (2026) to focus on extreme outcomes. Since IDR exhibits optimality properties with respect to weighted versions of the CRPS, the proposed approach inherits desirable properties of the PCRPS, and, in particular, facilitates fair comparisons between data-driven and physics-based models when forecasting extreme weather events. We apply this evaluation framework to forecasts in the WeatherBench 2 dataset issued by the AIWP models GraphCast, Pangu-Weather, and FuXi, with the ECMWF's high-resolution NWP model serving as a physics-based reference. The forecast models are compared when predicting mean sea level pressure, temperature, wind speed, and precipitation extremes, defined as exceedances or non-exceedances of thresholds obtained from historical observation data. We additionally study forecast performance when predicting record-breaking events, though the ordering of the different methods is largely insensitive to the thresholds on which emphasis is placed. We find that AIWP models, particularly FuXi, result in the most informative forecasts for extreme weather events across most settings, suggesting that AIWP models have the potential to outperform NWP models when forecasting extremes.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Biegert, T., Allen, S., Alber, A., & Lerch, S. (2026). Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2606.21170
MLA 9
Biegert, Tobias, et al. “Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2606.21170.
Chicago
Biegert, Tobias, Sam Allen, Annika Alber, and Sebastian Lerch. 2026. “Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2606.21170.
Harvard
Biegert, T. et al. (2026) ‘Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS’, arXiv (Cornell University). doi:10.48550/arxiv.2606.21170.
IEEE
T. Biegert, S. Allen, A. Alber, and S. Lerch, “Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2606.21170.
BibTeX
@misc{biegert2026towards,
title = {{Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS}},
author = {Biegert, Tobias and Allen, Sam and Alber, Annika and Lerch, Sebastian},
howpublished = {arXiv (Cornell University)},
year = {2026},
doi = {10.48550/arxiv.2606.21170},
url = {https://doi.org/10.48550/arxiv.2606.21170}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- numerical weather prediction
- extreme weather events
- probabilistic forecasting
- forecast evaluation
- GraphCast
- Pangu-Weather
- Fuxi
- ECMWF
Work with this paper
Most-cited papers in Earth and Planetary Sciences
- The ERA5 global reanalysisHans Hersbach, Bill Bell, Paul Berrisford and 40 more · Quarterly Journal of the Royal Meteorological Society · 2020 · 35,424 citations
- The NCEP/NCAR 40-Year Reanalysis ProjectEugenia Kalnay, Masao Kanamitsu, Robert E. Kistler and 19 more · Bulletin of the American Meteorological Society · 1996 · 28,486 citations
- Chemical and isotopic systematics of oceanic basalts: implications for mantle composition and processesShen‐Su Sun, W. F. McDonough · Geological Society London Special Publications · 1989 · 26,603 citations
- The ERA‐Interim reanalysis: configuration and performance of the data assimilation systemDick P. Dee, Sakari M. Uppala, A. J. Simmons and 33 more · Quarterly Journal of the Royal Meteorological Society · 2011 · 25,245 citations
- PAST: PALEONTOLOGICAL STATISTICAL SOFTWARE PACKAGE FOR EDUCATION AND DATA ANALYSISØyvind Hammer, David A. T. Harper, Paul D. Ryan · 2001 · 17,871 citations
Questions about this paper
How do I cite this paper in APA style?
Biegert, T., Allen, S., Alber, A., & Lerch, S. (2026). Towards Fair Comparisons of AI- and Physics-Based Weather Models for Extreme Events via the Weighted Potential CRPS. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2606.21170
How many times has this paper been cited?
3 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at arXiv (Cornell University).
Who wrote this paper?
Tobias Biegert, Sam Allen, Annika Alber, Sebastian Lerch.
What is the DOI of this paper?
10.48550/arxiv.2606.21170. The permanent link is https://doi.org/10.48550/arxiv.2606.21170.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.