Preprint · 2024 · Open access
WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models
arXiv (Cornell University) · Published 14 September 2024 · DOI 10.48550/arxiv.2409.09371
In brief
“WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models” is a 2024 preprint by Weixin Jin and 9 co-authors, published in arXiv (Cornell University). OpenAlex counts 16 citations as of 15 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 16
- Most cited in
- 2026 (10)
- References
- 29
- Authors
- 10
Abstract
In recent years, AI-based weather forecasting models have matched or even outperformed numerical weather prediction systems. However, most of these models have been trained and evaluated on reanalysis datasets like ERA5. These datasets, being products of numerical models, often diverge substantially from actual observations in some crucial variables like near-surface temperature, wind, precipitation and clouds - parameters that hold significant public interest. To address this divergence, we introduce WeatherReal, a novel benchmark dataset for weather forecasting, derived from global near-surface in-situ observations. WeatherReal also features a publicly accessible quality control and evaluation framework. This paper details the sources and processing methodologies underlying the dataset, and further illustrates the advantage of in-situ observations in capturing hyper-local and extreme weather through comparative analyses and case studies. Using WeatherReal, we evaluated several data-driven models and compared them with leading numerical models. Our work aims to advance the AI-based weather forecasting research towards a more application-focused and operation-ready approach.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Jin, W., Weyn, J. A., Zhao, P., Siqi, X., Bian, J., Fang, Z., Dong, H., Sun, H., Thambiratnam, K., & Zhang, Q. (2024). WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2409.09371
MLA 9
Jin, Weixin, et al. “WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models.” arXiv (Cornell University), 2024. https://doi.org/10.48550/arxiv.2409.09371.
Chicago
Jin, Weixin, Jonathan A. Weyn, Pengcheng Zhao, Xiang, Siqi, Jiang Bian, Zuliang Fang, Haiyu Dong, Hongyu Sun, Kit Thambiratnam, and Qi Zhang. 2024. “WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2409.09371.
Harvard
Jin, W. et al. (2024) ‘WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models’, arXiv (Cornell University). doi:10.48550/arxiv.2409.09371.
IEEE
W. Jin et al., “WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models,” arXiv (Cornell University), 2024, doi: 10.48550/arxiv.2409.09371.
BibTeX
@misc{jin2024weatherreal,
title = {{WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models}},
author = {Jin, Weixin and Weyn, Jonathan A. and Zhao, Pengcheng and Siqi, Xiang, and Bian, Jiang and Fang, Zuliang and Dong, Haiyu and Sun, Hongyu and Thambiratnam, Kit and Zhang, Qi},
howpublished = {arXiv (Cornell University)},
year = {2024},
doi = {10.48550/arxiv.2409.09371},
url = {https://doi.org/10.48550/arxiv.2409.09371}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2024 | 1 | |
| 2025 | 5 | |
| 2026 | 10 |
Topic and keywords
Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- in situ observation
- numerical weather prediction
- reanalysis data
- extreme weather events
Work with this paper
Most-cited papers in Earth and Planetary Sciences
- The ERA5 global reanalysisHans Hersbach, Bill Bell, Paul Berrisford and 40 more · Quarterly Journal of the Royal Meteorological Society · 2020 · 35,424 citations
- The NCEP/NCAR 40-Year Reanalysis ProjectEugenia Kalnay, Masao Kanamitsu, Robert E. Kistler and 19 more · Bulletin of the American Meteorological Society · 1996 · 28,486 citations
- Chemical and isotopic systematics of oceanic basalts: implications for mantle composition and processesShen‐Su Sun, W. F. McDonough · Geological Society London Special Publications · 1989 · 26,603 citations
- The ERA‐Interim reanalysis: configuration and performance of the data assimilation systemDick P. Dee, Sakari M. Uppala, A. J. Simmons and 33 more · Quarterly Journal of the Royal Meteorological Society · 2011 · 25,245 citations
- PAST: PALEONTOLOGICAL STATISTICAL SOFTWARE PACKAGE FOR EDUCATION AND DATA ANALYSISØyvind Hammer, David A. T. Harper, Paul D. Ryan · 2001 · 17,871 citations
Questions about this paper
How do I cite this paper in APA style?
Jin, W., Weyn, J. A., Zhao, P., Siqi, X., Bian, J., Fang, Z., Dong, H., Sun, H., Thambiratnam, K., & Zhang, Q. (2024). WeatherReal: A Benchmark Based on In-Situ Observations for Evaluating Weather Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2409.09371
How many times has this paper been cited?
16 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at arXiv (Cornell University).
Who wrote this paper?
Weixin Jin, Jonathan A. Weyn, Pengcheng Zhao, Xiang, Siqi, Jiang Bian, Zuliang Fang, Haiyu Dong, Hongyu Sun, Kit Thambiratnam, Qi Zhang.
What is the DOI of this paper?
10.48550/arxiv.2409.09371. The permanent link is https://doi.org/10.48550/arxiv.2409.09371.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.