Preprint · 2020 · Open access
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
arXiv (Cornell University) · Published 22 October 2020 · DOI 10.48550/arxiv.2010.11929
In brief
“An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale” is a 2020 preprint by Alexey Dosovitskiy and 11 co-authors, published in arXiv (Cornell University). OpenAlex counts 68,012 citations as of 18 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 68,012
- Most cited in
- 2025 (19,608)
- References
- 59
- Authors
- 12
Abstract
While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer applied directly to sequences of image patches can perform very well on image classification tasks. When pre-trained on large amounts of data and transferred to multiple mid-sized or small image recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc.), Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2010.11929
MLA 9
Dosovitskiy, Alexey, et al. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.” arXiv (Cornell University), 2020. https://doi.org/10.48550/arxiv.2010.11929.
Chicago
Dosovitskiy, Alexey, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, et al. 2020. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2010.11929.
Harvard
Dosovitskiy, A. et al. (2020) ‘An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale’, arXiv (Cornell University). doi:10.48550/arxiv.2010.11929.
IEEE
A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” arXiv (Cornell University), 2020, doi: 10.48550/arxiv.2010.11929.
BibTeX
@misc{dosovitskiy2020image,
title = {{An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale}},
author = {Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
howpublished = {arXiv (Cornell University)},
year = {2020},
doi = {10.48550/arxiv.2010.11929},
url = {https://doi.org/10.48550/arxiv.2010.11929}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2016 | 1 | |
| 2018 | 1 | |
| 2019 | 3 | |
| 2020 | 34 | |
| 2021 | 1,385 | |
| 2022 | 5,501 | |
| 2023 | 11,218 | |
| 2024 | 15,931 | |
| 2025 | 19,608 | |
| 2026 | 14,319 |
Topic and keywords
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- vision transformer
- image classification
- image patches
- attention mechanism
- pre-training
- transfer learning
- computational efficiency
- ImageNet
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2010.11929
How many times has this paper been cited?
68,012 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby.
What is the DOI of this paper?
10.48550/arxiv.2010.11929, permanently at https://doi.org/10.48550/arxiv.2010.11929. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.