Journal article · 2025 · Open access

A Practical Probabilistic Benchmark for AI Weather Models

Noah Brenowitz, Yair Cohen, Jaideep Pathak, Ankur Mahesh, Boris Bonev, Thorsten Kurth, Dale R. Durran, Peter Harrington, Michael S. Pritchard

Geophysical Research Letters, 52(7) · Published 8 April 2025 · DOI 10.1029/2024gl113656

Open PDFView at the publisherCite this paper

In brief

“A Practical Probabilistic Benchmark for AI Weather Models” is a 2025 journal article by Noah Brenowitz and 8 co-authors, published in Geophysical Research Letters. OpenAlex counts 69 citations as of 20 November 2026, which puts it in the top 3% of works published in 2025 by citations. It is open access under a CC BY licence: the full text is free to read at Geophysical Research Letters.

Citations
69
Most cited in
2026 (40)
Field-weighted impact
17.20
References
41
Authors
9

Abstract

Since the weather is chaotic, it is necessary to forecast an ensemble of future states. Recently, multiple AI weather models have emerged claiming breakthroughs in deterministic skill. Unfortunately, it is hard to fairly compare ensembles of AI forecasts because variations in ensembling methodology become confounding and the baseline data volume is immense. We address this by scoring lagged initial condition ensembles—whereby an ensemble can be constructed from a library of deterministic hindcasts. This allows the first parameter‐free intercomparison of leading AI weather models' probabilistic skill against an operational baseline. Lagged ensembles of the two leading AI weather models, GraphCast and Pangu, perform similarly even though the former outperforms the latter in deterministic scoring. These results are elaborated upon by sensitivity tests showing that commonly used multiple time‐step loss functions damage ensemble calibration.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Brenowitz, N., Cohen, Y., Pathak, J., Mahesh, A., Bonev, B., Kurth, T., Durran, D. R., Harrington, P., & Pritchard, M. S. (2025). A Practical Probabilistic Benchmark for AI Weather Models. Geophysical Research Letters, 52(7). https://doi.org/10.1029/2024gl113656

MLA 9

Brenowitz, Noah, et al. “A Practical Probabilistic Benchmark for AI Weather Models.” Geophysical Research Letters, vol. 52, no. 7, 2025. https://doi.org/10.1029/2024gl113656.

Chicago

Brenowitz, Noah, Yair Cohen, Jaideep Pathak, Ankur Mahesh, Boris Bonev, Thorsten Kurth, Dale R. Durran, Peter Harrington, and Michael S. Pritchard. 2025. “A Practical Probabilistic Benchmark for AI Weather Models.” Geophysical Research Letters 52 (7). https://doi.org/10.1029/2024gl113656.

Harvard

Brenowitz, N. et al. (2025) ‘A Practical Probabilistic Benchmark for AI Weather Models’, Geophysical Research Letters, 52(7). doi:10.1029/2024gl113656.

IEEE

N. Brenowitz et al., “A Practical Probabilistic Benchmark for AI Weather Models,” Geophysical Research Letters, vol. 52, no. 7, 2025, doi: 10.1029/2024gl113656.

BibTeX

@article{brenowitz2025practical,
  title     = {{A Practical Probabilistic Benchmark for AI Weather Models}},
  author    = {Brenowitz, Noah and Cohen, Yair and Pathak, Jaideep and Mahesh, Ankur and Bonev, Boris and Kurth, Thorsten and Durran, Dale R. and Harrington, Peter and Pritchard, Michael S.},
  journal   = {Geophysical Research Letters},
  year      = {2025},
  volume    = {52},
  number    = {7},
  doi       = {10.1029/2024gl113656},
  url       = {https://doi.org/10.1029/2024gl113656}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202466
20252323
20264040

Its field-weighted citation impact is 17.20: it has been cited 17.20 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations

  • probabilistic forecasting
  • ensemble forecasting
  • benchmarking
  • hindcasting
  • GraphCast
  • Pangu
  • ensemble calibration
  • deterministic forecasting

Work with this paper

Most-cited papers in Earth and Planetary Sciences

All highly cited Earth and Planetary Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Brenowitz, N., Cohen, Y., Pathak, J., Mahesh, A., Bonev, B., Kurth, T., Durran, D. R., Harrington, P., & Pritchard, M. S. (2025). A Practical Probabilistic Benchmark for AI Weather Models. Geophysical Research Letters, 52(7). https://doi.org/10.1029/2024gl113656

How many times has this paper been cited?

69 times, according to OpenAlex on 20 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Geophysical Research Letters: sign in and open the PDF from this page.

Who wrote this paper?

Noah Brenowitz, Yair Cohen, Jaideep Pathak, Ankur Mahesh, Boris Bonev, Thorsten Kurth, Dale R. Durran, Peter Harrington, Michael S. Pritchard.

What is the DOI of this paper?

10.1029/2024gl113656, permanently at https://doi.org/10.1029/2024gl113656. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 20 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.