Preprint · 2026 · Open access

TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling

Ruohan Wu, Ziqi Zhu, Yang Zhao, Jiarui Tang, Yingzhe Cui, Junshi Chen, Zhao Jing, Jun Shi, Hong An

arXiv (Cornell University) · Published 15 August 2026 · DOI 10.48550/arxiv.2608.15211

Free PDFView at the publisherCite this paper

In brief

“TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling” is a 2026 preprint by Ruohan Wu and 8 co-authors, published in arXiv (Cornell University). OpenAlex counts 0 citations as of 15 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
0
References
9
Authors
9

Abstract

Training high-resolution AI-based Earth forecasting models is memory-intensive. Window-based Swin Transformers reduce the quadratic cost of global attention, but existing distributed systems such as AERIS primarily target pixel-level models and do not jointly support convolutional sampling modules and shifted-window execution. Long-lead rollout finetuning further increases activation memory. To address these challenges, we present TERRA, a hierarchical parallel training framework for high-resolution Earth forecasting. TERRA introduces Sampling-Aware Window, Sequence, and Tensor Parallelism (SAWSTP), which preserves spatially contiguous layouts for sampling modules and routes tokens into topology-aware ragged window layouts for Transformer execution. For long-lead finetuning, Memory Orchestration (MO) provides rollout-aware checkpoint planning and combines input buffering with budget-constrained activation offloading. Experiments on the $1/12^\circ$ GLORYS-based Wenhai workload show that TERRA supports models with up to 11.4B parameters on 96 H200 GPUs and sustains up to $39.76$ PFLOPS, achieving $65.0\%$ strong-scaling and $94.1\%$ weak-scaling efficiency. Compared with checkpoint-only policies, MO further reduces peak allocated GPU memory by $32.2\%$--$51.8\%$ with at most $20.0\%$ step-time overhead, which makes finetuning with smaller patch sizes and longer rollouts feasible for improved forecasting accuracy.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Wu, R., Zhu, Z., Zhao, Y., Tang, J., Cui, Y., Chen, J., Jing, Z., Shi, J., & An, H. (2026). TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2608.15211

MLA 9

Wu, Ruohan, et al. “TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2608.15211.

Chicago

Wu, Ruohan, Ziqi Zhu, Yang Zhao, Jiarui Tang, Yingzhe Cui, Junshi Chen, Zhao Jing, Jun Shi, and Hong An. 2026. “TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2608.15211.

Harvard

Wu, R. et al. (2026) ‘TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling’, arXiv (Cornell University). doi:10.48550/arxiv.2608.15211.

IEEE

R. Wu et al., “TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2608.15211.

BibTeX

@misc{wu2026terra,
  title     = {{TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling}},
  author    = {Wu, Ruohan and Zhu, Ziqi and Zhao, Yang and Tang, Jiarui and Cui, Yingzhe and Chen, Junshi and Jing, Zhao and Shi, Jun and An, Hong},
  howpublished = {arXiv (Cornell University)},
  year      = {2026},
  doi       = {10.48550/arxiv.2608.15211},
  url       = {https://doi.org/10.48550/arxiv.2608.15211}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques

  • Swin Transformer
  • long-lead forecasting
  • checkpointing
  • tensor parallelism
  • strong and weak scaling

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Wu, R., Zhu, Z., Zhao, Y., Tang, J., Cui, Y., Chen, J., Jing, Z., Shi, J., & An, H. (2026). TERRA: A Hierarchical Parallel Training and Memory Orchestration Framework for High-Resolution AI-based Earth Modeling. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2608.15211

How many times has this paper been cited?

0 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at arXiv (Cornell University).

Who wrote this paper?

Ruohan Wu, Ziqi Zhu, Yang Zhao, Jiarui Tang, Yingzhe Cui, Junshi Chen, Zhao Jing, Jun Shi, Hong An.

What is the DOI of this paper?

10.48550/arxiv.2608.15211. The permanent link is https://doi.org/10.48550/arxiv.2608.15211.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.