Journal article · 2025 · Open access

Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions

Youshi Zeng, Yi‐Qing Ni, Zhengwei Chen, Guang-Zhi Zeng, Jia-Yao Wang, Pak Wai Chan

Engineering Applications of Computational Fluid Mechanics, 19(1) · Published 1 August 2025 · DOI 10.1080/19942060.2025.2538180

Open full textCite this paper

In brief

“Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions” is a 2025 journal article by Youshi Zeng and 5 co-authors, published in Engineering Applications of Computational Fluid Mechanics. OpenAlex counts 13 citations as of 20 November 2026, which puts it in the top 5% of works published in 2025 by citations. It is open access under a CC BY-NC licence: the full text is free to read at Engineering Applications of Computational Fluid Mechanics.

Citations
13
Most cited in
2026 (10)
Field-weighted impact
3.60
References
56
Authors
6

Abstract

Tropical cyclones (TCs) are destructive meteorological phenomena, necessitating accurate predictions of TC track and intensity to reduce risks to human life. This study evaluates three Transformer-based models – vanilla Transformer (Transformer), inverted Transformer (iTransformer), and temporal-variate Transformer (TVFormer) – which are trained, validated, and tested on best track data from 1980 to 2021 from the China Meteorological Administration for TC prediction, integrating temporal, variate, and hybrid token embeddings to analyze temporal and variate correlations. Comparative analysis with four recurrent neural network (RNN) models demonstrates the superiority of the refined Transformer models over RNNs: iTransformer reduces mean absolute error (MAE) and root mean square error (RMSE) by 29.55% and 25.80% (latitude), 50.31% and 46.18% (longitude), 8.71% and 9.98% (pressure), and 8.68% and 9.45% (wind speed), while TVFormer achieves MAE and RMSE reductions of 13.98% and 13.84% (latitude), 39.11% and 38.02% (longitude), 13.69% and 14.02% (pressure), and 12.84% and 12.94% (wind speed) on average. Among Transformer variants, iTransformer excels in track prediction, outperforming Transformer with 21.74% lower MAE and 18.26% lower RMSE for latitude, and 32.73% lower MAE and 24.01% lower RMSE for longitude. TVFormer dominates intensity prediction, reducing pressure errors by 4.42% (MAE) and 3.92% (RMSE) and wind speed errors by 19.21% (MAE) and 14.79% (RMSE) compared to Transformer, while outperforming iTransformer with 4.59% lower MAE and 3.68% lower RMSE for pressure and 3.83% lower MAE and 3.18% lower RMSE for wind speed. Notably, TVFormer also enhances track prediction, with 7.10% reduction in MAE and 7.02% reduction in RMSE for latitude, and 22.84% reduction in MAE and 17.09% reduction in RMSE for longitude compared to Transformer. These results highlight the superiority of iTransformer in track prediction and the efficacy of TVFormer in intensity prediction, thanks to their ability to exploit temporal and variate dependencies, offering potential for TC disaster preparedness systems.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Zeng, Y., Ni, Y., Chen, Z., Zeng, G.-Z., Wang, J.-Y., & Chan, P. W. (2025). Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions. Engineering Applications of Computational Fluid Mechanics, 19(1). https://doi.org/10.1080/19942060.2025.2538180

MLA 9

Zeng, Youshi, et al. “Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions.” Engineering Applications of Computational Fluid Mechanics, vol. 19, no. 1, 2025. https://doi.org/10.1080/19942060.2025.2538180.

Chicago

Zeng, Youshi, Yi‐Qing Ni, Zhengwei Chen, Guang-Zhi Zeng, Jia-Yao Wang, and Pak Wai Chan. 2025. “Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions.” Engineering Applications of Computational Fluid Mechanics 19 (1). https://doi.org/10.1080/19942060.2025.2538180.

Harvard

Zeng, Y. et al. (2025) ‘Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions’, Engineering Applications of Computational Fluid Mechanics, 19(1). doi:10.1080/19942060.2025.2538180.

IEEE

Y. Zeng, Y. Ni, Z. Chen, G.-Z. Zeng, J.-Y. Wang, and P. W. Chan, “Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions,” Engineering Applications of Computational Fluid Mechanics, vol. 19, no. 1, 2025, doi: 10.1080/19942060.2025.2538180.

BibTeX

@article{zeng2025investigation,
  title     = {{Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions}},
  author    = {Zeng, Youshi and Ni, Yi‐Qing and Chen, Zhengwei and Zeng, Guang-Zhi and Wang, Jia-Yao and Chan, Pak Wai},
  journal   = {Engineering Applications of Computational Fluid Mechanics},
  year      = {2025},
  volume    = {19},
  number    = {1},
  doi       = {10.1080/19942060.2025.2538180},
  url       = {https://doi.org/10.1080/19942060.2025.2538180}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Tropical and Extratropical Cyclones Research

  • tropical cyclones
  • tropical cyclone track forecasting
  • tropical cyclone intensity forecasting
  • transformer models
  • token embeddings
  • inverted transformer
  • best track data
  • wind speed

Work with this paper

Most-cited papers in Earth and Planetary Sciences

All highly cited Earth and Planetary Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Zeng, Y., Ni, Y., Chen, Z., Zeng, G.-Z., Wang, J.-Y., & Chan, P. W. (2025). Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions. Engineering Applications of Computational Fluid Mechanics, 19(1). https://doi.org/10.1080/19942060.2025.2538180

How many times has this paper been cited?

13 times, according to OpenAlex on 20 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Engineering Applications of Computational Fluid Mechanics: sign in and open the full text from this page.

Who wrote this paper?

Youshi Zeng, Yi‐Qing Ni, Zhengwei Chen, Guang-Zhi Zeng, Jia-Yao Wang, Pak Wai Chan.

What is the DOI of this paper?

10.1080/19942060.2025.2538180, permanently at https://doi.org/10.1080/19942060.2025.2538180. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 20 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.