Preprint · 2020 · Open access

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby

arXiv (Cornell University) · Published 22 October 2020 · DOI 10.48550/arxiv.2010.11929

Open PDFView at the publisherCite this paper

In brief

“An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale” is a 2020 preprint by Alexey Dosovitskiy and 11 co-authors, published in arXiv (Cornell University). OpenAlex counts 68,012 citations as of 18 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
68,012
Most cited in
2025 (19,608)
References
59
Authors
12

Abstract

While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer applied directly to sequences of image patches can perform very well on image classification tasks. When pre-trained on large amounts of data and transferred to multiple mid-sized or small image recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc.), Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2010.11929

MLA 9

Dosovitskiy, Alexey, et al. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.” arXiv (Cornell University), 2020. https://doi.org/10.48550/arxiv.2010.11929.

Chicago

Dosovitskiy, Alexey, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, et al. 2020. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2010.11929.

Harvard

Dosovitskiy, A. et al. (2020) ‘An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale’, arXiv (Cornell University). doi:10.48550/arxiv.2010.11929.

IEEE

A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” arXiv (Cornell University), 2020, doi: 10.48550/arxiv.2010.11929.

BibTeX

@misc{dosovitskiy2020image,
  title     = {{An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale}},
  author    = {Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
  howpublished = {arXiv (Cornell University)},
  year      = {2020},
  doi       = {10.48550/arxiv.2010.11929},
  url       = {https://doi.org/10.48550/arxiv.2010.11929}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
201611
201811
201933
20203434
20211,3851385
20225,5015501
202311,21811218
202415,93115931
202519,60819608
202614,31914319

Topic and keywords

Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications

  • vision transformer
  • image classification
  • image patches
  • attention mechanism
  • pre-training
  • transfer learning
  • computational efficiency
  • ImageNet

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2020). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2010.11929

How many times has this paper been cited?

68,012 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby.

What is the DOI of this paper?

10.48550/arxiv.2010.11929, permanently at https://doi.org/10.48550/arxiv.2010.11929. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.