Journal article · 2025 · Open access
Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions
Engineering Applications of Computational Fluid Mechanics, 19(1) · Published 1 August 2025 · DOI 10.1080/19942060.2025.2538180
In brief
“Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions” is a 2025 journal article by Youshi Zeng and 5 co-authors, published in Engineering Applications of Computational Fluid Mechanics. OpenAlex counts 13 citations as of 20 November 2026, which puts it in the top 5% of works published in 2025 by citations. It is open access under a CC BY-NC licence: the full text is free to read at Engineering Applications of Computational Fluid Mechanics.
- Citations
- 13
- Most cited in
- 2026 (10)
- Field-weighted impact
- 3.60
- References
- 56
- Authors
- 6
Abstract
Tropical cyclones (TCs) are destructive meteorological phenomena, necessitating accurate predictions of TC track and intensity to reduce risks to human life. This study evaluates three Transformer-based models – vanilla Transformer (Transformer), inverted Transformer (iTransformer), and temporal-variate Transformer (TVFormer) – which are trained, validated, and tested on best track data from 1980 to 2021 from the China Meteorological Administration for TC prediction, integrating temporal, variate, and hybrid token embeddings to analyze temporal and variate correlations. Comparative analysis with four recurrent neural network (RNN) models demonstrates the superiority of the refined Transformer models over RNNs: iTransformer reduces mean absolute error (MAE) and root mean square error (RMSE) by 29.55% and 25.80% (latitude), 50.31% and 46.18% (longitude), 8.71% and 9.98% (pressure), and 8.68% and 9.45% (wind speed), while TVFormer achieves MAE and RMSE reductions of 13.98% and 13.84% (latitude), 39.11% and 38.02% (longitude), 13.69% and 14.02% (pressure), and 12.84% and 12.94% (wind speed) on average. Among Transformer variants, iTransformer excels in track prediction, outperforming Transformer with 21.74% lower MAE and 18.26% lower RMSE for latitude, and 32.73% lower MAE and 24.01% lower RMSE for longitude. TVFormer dominates intensity prediction, reducing pressure errors by 4.42% (MAE) and 3.92% (RMSE) and wind speed errors by 19.21% (MAE) and 14.79% (RMSE) compared to Transformer, while outperforming iTransformer with 4.59% lower MAE and 3.68% lower RMSE for pressure and 3.83% lower MAE and 3.18% lower RMSE for wind speed. Notably, TVFormer also enhances track prediction, with 7.10% reduction in MAE and 7.02% reduction in RMSE for latitude, and 22.84% reduction in MAE and 17.09% reduction in RMSE for longitude compared to Transformer. These results highlight the superiority of iTransformer in track prediction and the efficacy of TVFormer in intensity prediction, thanks to their ability to exploit temporal and variate dependencies, offering potential for TC disaster preparedness systems.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Zeng, Y., Ni, Y., Chen, Z., Zeng, G.-Z., Wang, J.-Y., & Chan, P. W. (2025). Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions. Engineering Applications of Computational Fluid Mechanics, 19(1). https://doi.org/10.1080/19942060.2025.2538180
MLA 9
Zeng, Youshi, et al. “Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions.” Engineering Applications of Computational Fluid Mechanics, vol. 19, no. 1, 2025. https://doi.org/10.1080/19942060.2025.2538180.
Chicago
Zeng, Youshi, Yi‐Qing Ni, Zhengwei Chen, Guang-Zhi Zeng, Jia-Yao Wang, and Pak Wai Chan. 2025. “Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions.” Engineering Applications of Computational Fluid Mechanics 19 (1). https://doi.org/10.1080/19942060.2025.2538180.
Harvard
Zeng, Y. et al. (2025) ‘Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions’, Engineering Applications of Computational Fluid Mechanics, 19(1). doi:10.1080/19942060.2025.2538180.
IEEE
Y. Zeng, Y. Ni, Z. Chen, G.-Z. Zeng, J.-Y. Wang, and P. W. Chan, “Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions,” Engineering Applications of Computational Fluid Mechanics, vol. 19, no. 1, 2025, doi: 10.1080/19942060.2025.2538180.
BibTeX
@article{zeng2025investigation,
title = {{Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions}},
author = {Zeng, Youshi and Ni, Yi‐Qing and Chen, Zhengwei and Zeng, Guang-Zhi and Wang, Jia-Yao and Chan, Pak Wai},
journal = {Engineering Applications of Computational Fluid Mechanics},
year = {2025},
volume = {19},
number = {1},
doi = {10.1080/19942060.2025.2538180},
url = {https://doi.org/10.1080/19942060.2025.2538180}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Tropical and Extratropical Cyclones Research
- tropical cyclones
- tropical cyclone track forecasting
- tropical cyclone intensity forecasting
- transformer models
- token embeddings
- inverted transformer
- best track data
- wind speed
Work with this paper
Most-cited papers in Earth and Planetary Sciences
- The ERA5 global reanalysisHans Hersbach, Bill Bell, Paul Berrisford and 40 more · Quarterly Journal of the Royal Meteorological Society · 2020 · 35,424 citations
- The NCEP/NCAR 40-Year Reanalysis ProjectEugenia Kalnay, Masao Kanamitsu, Robert E. Kistler and 19 more · Bulletin of the American Meteorological Society · 1996 · 28,486 citations
- Chemical and isotopic systematics of oceanic basalts: implications for mantle composition and processesShen‐Su Sun, W. F. McDonough · Geological Society London Special Publications · 1989 · 26,603 citations
- The ERA‐Interim reanalysis: configuration and performance of the data assimilation systemDick P. Dee, Sakari M. Uppala, A. J. Simmons and 33 more · Quarterly Journal of the Royal Meteorological Society · 2011 · 25,245 citations
- PAST: PALEONTOLOGICAL STATISTICAL SOFTWARE PACKAGE FOR EDUCATION AND DATA ANALYSISØyvind Hammer, David A. T. Harper, Paul D. Ryan · 2001 · 17,871 citations
Questions about this paper
How do I cite this paper in APA style?
Zeng, Y., Ni, Y., Chen, Z., Zeng, G.-Z., Wang, J.-Y., & Chan, P. W. (2025). Investigation of the impact of token embeddings in Transformer-based models on short-term tropical cyclone track and intensity predictions. Engineering Applications of Computational Fluid Mechanics, 19(1). https://doi.org/10.1080/19942060.2025.2538180
How many times has this paper been cited?
13 times, according to OpenAlex on 20 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Engineering Applications of Computational Fluid Mechanics: sign in and open the full text from this page.
Who wrote this paper?
Youshi Zeng, Yi‐Qing Ni, Zhengwei Chen, Guang-Zhi Zeng, Jia-Yao Wang, Pak Wai Chan.
What is the DOI of this paper?
10.1080/19942060.2025.2538180, permanently at https://doi.org/10.1080/19942060.2025.2538180. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 20 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.