Paper · 2026 · Open access

ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?

Mosab Hawarey

AIR Journal of Mathematics and Computational Sciences, 2026 · Published 4 March 2026 · DOI 10.65737/airmcs2026405

Read the free full textCite this paper

In brief

“ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?” is a 2026 paper by Mosab Hawarey, published in AIR Journal of Mathematics and Computational Sciences. OpenAlex counts 7 citations as of 15 November 2026, which puts it in the top 2% of works published in 2026 by citations. It is open access under a CC BY licence: the full text is free to read.

Citations
7
Most cited in
2026 (7)
Field-weighted impact
11.39
References
24
Authors
1

Abstract

Climate foundation models (FMs) have achieved remarkable success in weather forecasting, yet exhibit puzzling performance gaps between tasks: deterministic field prediction rivals operational numerical weather prediction, while extreme event detection lags significantly behind. We provide a theoretical explanation through the lens of in-context learning (ICL). Extending the ICL characterization framework to spatiotemporal climate data, we prove the Climate Prediction Dichotomy Theorem: every natural climate task falls into exactly one of two complexity categories. Type A (ICL-Easy) tasks—including temperature, pressure, and wind field prediction—admit additive sufficient statistics enabling attention-based computation with sample complexity nICL = Θ(nERM). Type C (ICL-Hard) tasks—including extreme event detection, tipping point identification, and compound event localization—require combinatorial sufficient statistics that provably exceed the computational capacity of constant-depth polynomial-size transformers when the number of simultaneous events J exceeds a threshold J* = O(log log n) ≈ 3–5. We establish the predictability horizon constraint: weather forecasting is ICL-Easy for lead times τ < τL ≈ 14 days, while climate statistics remain ICL-accessible but individual trajectories are fundamentally unpredictable. Our analysis yields six testable predictions about climate FM behavior and five deployment guidelines distinguishing when ICL suffices versus when fine-tuning is required. The dichotomy provides a principled foundation for understanding why Pangu-Weather and GraphCast excel at field prediction while struggling with event detection—and guides the design of next-generation climate AI systems.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Hawarey, M. (2026). ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate? AIR Journal of Mathematics and Computational Sciences, 2026. https://doi.org/10.65737/airmcs2026405

MLA 9

Hawarey, Mosab. “ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?” AIR Journal of Mathematics and Computational Sciences, vol. 2026, 2026. https://doi.org/10.65737/airmcs2026405.

Chicago

Hawarey, Mosab. 2026. “ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?” AIR Journal of Mathematics and Computational Sciences 2026. https://doi.org/10.65737/airmcs2026405.

Harvard

Hawarey, M. (2026) ‘ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?’, AIR Journal of Mathematics and Computational Sciences, 2026. doi:10.65737/airmcs2026405.

IEEE

M. Hawarey, “ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?,” AIR Journal of Mathematics and Computational Sciences, vol. 2026, 2026, doi: 10.65737/airmcs2026405.

BibTeX

@article{hawarey2026characterization,
  title     = {{ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate?}},
  author    = {Hawarey, Mosab},
  journal   = {AIR Journal of Mathematics and Computational Sciences},
  year      = {2026},
  volume    = {2026},
  doi       = {10.65737/airmcs2026405},
  url       = {https://doi.org/10.65737/airmcs2026405}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations

  • in-context learning
  • transformer models
  • extreme event detection
  • tipping points
  • compound events
  • predictability horizon
  • sufficient statistics
  • GraphCast

Work with this paper

Most-cited papers in Earth and Planetary Sciences

All highly cited Earth and Planetary Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Hawarey, M. (2026). ICL Characterization of Climate Foundation Models: When Can Transformers Learn Weather and Climate? AIR Journal of Mathematics and Computational Sciences, 2026. https://doi.org/10.65737/airmcs2026405

How many times has this paper been cited?

7 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text.

Who wrote this paper?

Mosab Hawarey.

What is the DOI of this paper?

10.65737/airmcs2026405. The permanent link is https://doi.org/10.65737/airmcs2026405.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.