Conference paper · 2023 · Open access
Automated Program Repair in the Era of Large Pre-trained Language Models
ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 1482–1494 · Published 1 May 2023 · DOI 10.1109/icse48619.2023.00129
In brief
“Automated Program Repair in the Era of Large Pre-trained Language Models” is a 2023 conference paper by Chunqiu Steven Xia and 2 co-authors, published in ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). OpenAlex counts 883 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 883
- Most cited in
- 2026 (314)
- Field-weighted impact
- 111.15
- References
- 75
- Authors
- 3
Abstract
Automated Program Repair (APR) aims to help developers automatically patch software bugs. However, current state-of-the-art traditional and learning-based APR techniques face the problem of limited patch variety, failing to fix complicated bugs. This is mainly due to the reliance on bug-fixing datasets to craft fix templates (traditional) or directly predict potential patches (learning-based). Large Pre-Trained Language Models (LLMs), trained using billions of text/code tokens, can potentially help avoid this issue. Very recently, researchers have directly leveraged LLMs for APR without relying on any bug-fixing datasets. Meanwhile, such existing work either failed to include state-of-the-art LLMs or was not evaluated on realistic datasets. Thus, the true power of modern LLMs on the important APR problem is yet to be revealed. In this work, we perform the first extensive study on directly applying LLMs for APR. We select 9 recent state-of-the-art LLMs, including both generative and infilling models, ranging from 125M to 20B in size. We designed 3 different repair settings to evaluate the different ways we can use LLMs to generate patches: 1) generate the entire patch function, 2) fill in a chunk of code given the prefix and suffix 3) output a single line fix. We apply the LLMs under these repair settings on 5 datasets across 3 different languages and compare different LLMs in the number of bugs fixed, generation speed and compilation rate. We also compare the LLMs against recent state-of-the-art APR tools. Our study demonstrates that directly applying state-of-the-art LLMs can already substantially outperform all existing APR techniques on all our datasets. Among the studied LLMs, the scaling effect exists for APR where larger models tend to achieve better performance. Also, we show for the first time that suffix code after the buggy line (adopted in infilling-style APR) is important in not only generating more fixes but more patches with higher compilation rate. Besides patch generation, the LLMs consider correct patches to be more natural than other ones, and can even be leveraged for effective patch ranking or patch correctness checking. Lastly, we show that LLM-based APR can be further substantially boosted via: 1) increasing the sample size, and 2) incorporating fix template information.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Xia, C. S., Wei, Y., & Zhang, L. (2023). Automated Program Repair in the Era of Large Pre-trained Language Models. ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 1482–1494. https://doi.org/10.1109/icse48619.2023.00129
MLA 9
Xia, Chunqiu Steven, et al. “Automated Program Repair in the Era of Large Pre-trained Language Models.” ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2023, pp. 1482–1494. https://doi.org/10.1109/icse48619.2023.00129.
Chicago
Xia, Chunqiu Steven, Yuxiang Wei, and Lingming Zhang. 2023. “Automated Program Repair in the Era of Large Pre-trained Language Models.” ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion): 1482–1494. https://doi.org/10.1109/icse48619.2023.00129.
Harvard
Xia, C. S., Wei, Y. and Zhang, L. (2023) ‘Automated Program Repair in the Era of Large Pre-trained Language Models’, ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 1482–1494. doi:10.1109/icse48619.2023.00129.
IEEE
C. S. Xia, Y. Wei, and L. Zhang, “Automated Program Repair in the Era of Large Pre-trained Language Models,” ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 1482–1494, 2023, doi: 10.1109/icse48619.2023.00129.
BibTeX
@inproceedings{xia2023automated,
title = {{Automated Program Repair in the Era of Large Pre-trained Language Models}},
author = {Xia, Chunqiu Steven and Wei, Yuxiang and Zhang, Lingming},
booktitle = {ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)},
year = {2023},
pages = {1482--1494},
doi = {10.1109/icse48619.2023.00129},
url = {https://doi.org/10.1109/icse48619.2023.00129}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2013 | 1 | |
| 2022 | 2 | |
| 2023 | 60 | |
| 2024 | 228 | |
| 2025 | 278 | |
| 2026 | 314 |
Its field-weighted citation impact is 111.15: it has been cited 111.15 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- automated program repair
- large pre-trained language models
- bug fixes
- patch generation
- infilling
- model scaling
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Xia, C. S., Wei, Y., & Zhang, L. (2023). Automated Program Repair in the Era of Large Pre-trained Language Models. ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 1482–1494. https://doi.org/10.1109/icse48619.2023.00129
How many times has this paper been cited?
883 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Chunqiu Steven Xia, Yuxiang Wei, Lingming Zhang.
What is the DOI of this paper?
10.1109/icse48619.2023.00129, permanently at https://doi.org/10.1109/icse48619.2023.00129. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.