Preprint · 2026 · Open access
TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling
arXiv (Cornell University) · Published 15 August 2026 · DOI 10.48550/arxiv.2608.15211
In brief
“TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling” is a 2026 preprint by Ruohan Wu and 8 co-authors, published in arXiv (Cornell University). OpenAlex counts 0 citations as of 15 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 0
- References
- 9
- Authors
- 9
Abstract
Training high-resolution AI-based Earth forecasting models is memory-intensive. Window-based Swin Transformers reduce the quadratic cost of global attention, but existing distributed systems such as AERIS primarily target pixel-level models and do not jointly support convolutional sampling modules and shifted-window execution. Long-lead rollout finetuning further increases activation memory. To address these challenges, we present TERRA, a hierarchical parallel training framework for high-resolution Earth forecasting. TERRA introduces Sampling-Aware Window, Sequence, and Tensor Parallelism (SAWSTP), which preserves spatially contiguous layouts for sampling modules and routes tokens into topology-aware ragged window layouts for Transformer execution. For long-lead finetuning, Memory Orchestration (MO) provides rollout-aware checkpoint planning and combines input buffering with budget-constrained activation offloading. Experiments on the $1/12^\circ$ GLORYS-based Wenhai workload show that TERRA supports models with up to 11.4B parameters on 96 H200 GPUs and sustains up to $39.76$ PFLOPS, achieving $65.0\%$ strong-scaling and $94.1\%$ weak-scaling efficiency. Compared with checkpoint-only policies, MO further reduces peak allocated GPU memory by $32.2\%$--$51.8\%$ with at most $20.0\%$ step-time overhead, which makes finetuning with smaller patch sizes and longer rollouts feasible for improved forecasting accuracy.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Wu, R., Zhu, Z., Zhao, Y., Tang, J., Cui, Y., Chen, J., Jing, Z., Shi, J., & An, H. (2026). TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2608.15211
MLA 9
Wu, Ruohan, et al. “TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2608.15211.
Chicago
Wu, Ruohan, Ziqi Zhu, Yang Zhao, Jiarui Tang, Yingzhe Cui, Junshi Chen, Zhao Jing, Jun Shi, and Hong An. 2026. “TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2608.15211.
Harvard
Wu, R. et al. (2026) ‘TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling’, arXiv (Cornell University). doi:10.48550/arxiv.2608.15211.
IEEE
R. Wu et al., “TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2608.15211.
BibTeX
@misc{wu2026terra,
title = {{TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling}},
author = {Wu, Ruohan and Zhu, Ziqi and Zhao, Yang and Tang, Jiarui and Cui, Yingzhe and Chen, Junshi and Jing, Zhao and Shi, Jun and An, Hong},
howpublished = {arXiv (Cornell University)},
year = {2026},
doi = {10.48550/arxiv.2608.15211},
url = {https://doi.org/10.48550/arxiv.2608.15211}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Swin Transformer
- long-lead forecasting
- checkpointing
- tensor parallelism
- strong and weak scaling
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Wu, R., Zhu, Z., Zhao, Y., Tang, J., Cui, Y., Chen, J., Jing, Z., Shi, J., & An, H. (2026). TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2608.15211
How many times has this paper been cited?
0 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at arXiv (Cornell University).
Who wrote this paper?
Ruohan Wu, Ziqi Zhu, Yang Zhao, Jiarui Tang, Yingzhe Cui, Junshi Chen, Zhao Jing, Jun Shi, Hong An.
What is the DOI of this paper?
10.48550/arxiv.2608.15211. The permanent link is https://doi.org/10.48550/arxiv.2608.15211.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.