Preprint · 2026 · Open access

RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges

Ruize Li, Zhibin Wen, Tao Han, Hao Chen, Fenghua Ling, Wei Zhang, Song Guo, Lei Bai

arXiv (Cornell University) · Published 24 May 2026 · DOI 10.48550/arxiv.2605.24945

Free PDFView at the publisherCite this paper

In brief

“RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges” is a 2026 preprint by Ruize Li and 7 co-authors, published in arXiv (Cornell University). OpenAlex counts 0 citations as of 15 November 2026. It is open access under a CC BY-NC-SA licence: the full text is free to read at arXiv (Cornell University).

Citations
0
References
17
Authors
8

Abstract

Accurate evaluation of weather forecasting models is critical for their reliable deployment in real-world applications. However, existing benchmarks predominantly rely on reanalysis products such as ERA5, which are generated through delayed data assimilation and do not reflect the constraints of real-time operational forecasting, thereby resulting in a systematic mismatch between benchmark performance and real-world forecasting. In this work, we introduce RealBench, a next-generation benchmark for AI weather forecasting that emphasizes realistic evaluation under operational conditions. RealBench features a strictly out-of-distribution test set spanning 2025 to eliminate data leakage and capture recent atmospheric regimes. It integrates multiple data sources, including low-latency operational analysis and a large-scale global in-situ observation dataset comprising over 10,000 stations, enabling direct evaluation against real atmospheric measurements. Beyond standard global metrics, RealBench provides a comprehensive evaluation framework for high-impact extreme events, including heatwaves, cold surges, and tropical cyclones, using event-specific metrics that better reflect real-world forecasting priorities. The evaluation results reveal substantial discrepancies between reanalysis-based metrics and real-world performance, particularly concerning extreme events. By highlighting the limitations of existing benchmarks, this work establishes a more faithful and operationally relevant evaluation paradigm, providing a rigorous foundation for advancing next-generation AI weather forecasting systems. The benchmark implementation is available at: https://github.com/lixruize-del/NWP-Benchmark.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Li, R., Wen, Z., Han, T., Chen, H., Ling, F., Zhang, W., Guo, S., & Bai, L. (2026). RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.24945

MLA 9

Li, Ruize, et al. “RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2605.24945.

Chicago

Li, Ruize, Zhibin Wen, Tao Han, Hao Chen, Fenghua Ling, Wei Zhang, Song Guo, and Lei Bai. 2026. “RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.24945.

Harvard

Li, R. et al. (2026) ‘RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges’, arXiv (Cornell University). doi:10.48550/arxiv.2605.24945.

IEEE

R. Li et al., “RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2605.24945.

BibTeX

@misc{li2026realbench,
  title     = {{RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges}},
  author    = {Li, Ruize and Wen, Zhibin and Han, Tao and Chen, Hao and Ling, Fenghua and Zhang, Wei and Guo, Song and Bai, Lei},
  howpublished = {arXiv (Cornell University)},
  year      = {2026},
  doi       = {10.48550/arxiv.2605.24945},
  url       = {https://doi.org/10.48550/arxiv.2605.24945}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations

  • numerical weather prediction
  • benchmarking
  • operational forecasting
  • out-of-distribution evaluation
  • in situ observation
  • extreme weather events
  • heat waves
  • cold surges

Work with this paper

Most-cited papers in Earth and Planetary Sciences

All highly cited Earth and Planetary Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Li, R., Wen, Z., Han, T., Chen, H., Ling, F., Zhang, W., Guo, S., & Bai, L. (2026). RealBench: Benchmarking Data-Driven Numerical Weather Forecasting Under Operational Conditions and Extreme Event Challenges. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.24945

How many times has this paper been cited?

0 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at arXiv (Cornell University).

Who wrote this paper?

Ruize Li, Zhibin Wen, Tao Han, Hao Chen, Fenghua Ling, Wei Zhang, Song Guo, Lei Bai.

What is the DOI of this paper?

10.48550/arxiv.2605.24945. The permanent link is https://doi.org/10.48550/arxiv.2605.24945.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.