Journal article · 2026 · Open access
Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions
IEEE Transactions on Software Engineering, 52(7), 2111–2126 · Published 29 April 2026 · DOI 10.1109/tse.2026.3688237
In brief
“Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions” is a 2026 journal article by Kexin Sun and 8 co-authors, published in IEEE Transactions on Software Engineering. OpenAlex counts 24 citations as of 17 November 2026, which puts it in the top 7% of works published in 2026 by citations. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 24
- Most cited in
- 2026 (22)
- Field-weighted impact
- 56.98
- References
- 48
- Authors
- 9
Abstract
AI-based code review tools automatically review and comment on pull requests to improve code quality. Despite their growing presence, little is known about their actual impact. We present a large-scale empirical study of 16 popular AI-based code review actions for GitHub workflows, analyzing more than 22,000 review comments in 178 repositories. We investigate (1) how these tools are adopted and configured, (2) whether their comments lead to code changes, and (3) which factors influence their effectiveness. We develop a two-stage LLM-assisted framework to determine whether review comments are addressed, and use interpretable machine learning to identify influencing factors. Our findings show that, while adoption is growing, effectiveness varies widely. Comments that are concise, contain code snippets, and are manually triggered, particularly those from hunk-level review tools, are more likely to result in code changes. These results highlight the importance of careful tool design and suggest directions for improving AI-based code review systems.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Sun, K., Kuang, H., Baltes, S., Zhou, X., Zhang, H., Ma, X., Rong, G., Shao, D., & Treude, C. (2026). Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions. IEEE Transactions on Software Engineering, 52(7), 2111–2126. https://doi.org/10.1109/tse.2026.3688237
MLA 9
Sun, Kexin, et al. “Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions.” IEEE Transactions on Software Engineering, vol. 52, no. 7, 2026, pp. 2111–2126. https://doi.org/10.1109/tse.2026.3688237.
Chicago
Sun, Kexin, Hongyu Kuang, Sebastian Baltes, Xin Zhou, He Zhang, Xiaoxing Ma, Guoping Rong, Dong Shao, and Christoph Treude. 2026. “Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions.” IEEE Transactions on Software Engineering 52 (7): 2111–2126. https://doi.org/10.1109/tse.2026.3688237.
Harvard
Sun, K. et al. (2026) ‘Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions’, IEEE Transactions on Software Engineering, 52(7), pp. 2111–2126. doi:10.1109/tse.2026.3688237.
IEEE
K. Sun et al., “Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions,” IEEE Transactions on Software Engineering, vol. 52, no. 7, pp. 2111–2126, 2026, doi: 10.1109/tse.2026.3688237.
BibTeX
@article{sun2026does,
title = {{Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions}},
author = {Sun, Kexin and Kuang, Hongyu and Baltes, Sebastian and Zhou, Xin and Zhang, He and Ma, Xiaoxing and Rong, Guoping and Shao, Dong and Treude, Christoph},
journal = {IEEE Transactions on Software Engineering},
year = {2026},
volume = {52},
number = {7},
pages = {2111--2126},
doi = {10.1109/tse.2026.3688237},
url = {https://doi.org/10.1109/tse.2026.3688237}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- GitHub Actions
- code amendments
- model interpretability
- tool adoption
- code quality
- tool effectiveness
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Sun, K., Kuang, H., Baltes, S., Zhou, X., Zhang, H., Ma, X., Rong, G., Shao, D., & Treude, C. (2026). Does AI Code Review Lead to Code Changes? A Case Study of GitHub Actions. IEEE Transactions on Software Engineering, 52(7), 2111–2126. https://doi.org/10.1109/tse.2026.3688237
How many times has this paper been cited?
24 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Kexin Sun, Hongyu Kuang, Sebastian Baltes, Xin Zhou, He Zhang, Xiaoxing Ma, Guoping Rong, Dong Shao, Christoph Treude.
What is the DOI of this paper?
10.1109/tse.2026.3688237, permanently at https://doi.org/10.1109/tse.2026.3688237. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.