Preprint · 2026 · Open access
LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models
arXiv (Cornell University) · Published 24 June 2026 · DOI 10.48550/arxiv.2606.25402
In brief
“LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models” is a 2026 preprint by Daniele Cipollone and 4 co-authors, published in arXiv (Cornell University). OpenAlex counts 1 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 1
- Most cited in
- 2026 (1)
- References
- 17
- Authors
- 5
Abstract
Large software projects often depend on older versions of libraries, even as APIs continue to evolve across releases. This creates a challenge for LLMs: they must maintain knowledge of multiple API versions, not merely the latest or most common one. However, current LLMs are trained on temporally mixed corpora and lack explicit mechanisms for such version-specific reasoning, leading to anachronistic errors - calling APIs as they exist in a different library version. To systematically evaluate this phenomenon, we introduce LibEvoBench, a multi-task benchmark spanning multiple versions of widely used Python libraries, along with a new metric, the Software Evolution Understanding Score (SEUS), to measure models' consistency when working with evolving APIs. Our results show that state-of-the-art models are largely version-oblivious: performance degrades for evolving APIs, while for stable APIs it remains the same across versions. Moreover, simply specifying the target version provides no benefit, while relevant documentation significantly boosts models' accuracy. These findings highlight a systematic limitation of current training paradigms and motivate new approaches for temporally grounded knowledge in code generation.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Cipollone, D., Titov, S., Izadi, M., Bogomolov, E., & van Deursen, A. (2026). LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2606.25402
MLA 9
Cipollone, Daniele, et al. “LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2606.25402.
Chicago
Cipollone, Daniele, Sergey Titov, Maliheh Izadi, Egor Bogomolov, and Arie van Deursen. 2026. “LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2606.25402.
Harvard
Cipollone, D. et al. (2026) ‘LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models’, arXiv (Cornell University). doi:10.48550/arxiv.2606.25402.
IEEE
D. Cipollone, S. Titov, M. Izadi, E. Bogomolov, and A. van Deursen, “LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2606.25402.
BibTeX
@misc{cipollone2026libevobench,
title = {{LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models}},
author = {Cipollone, Daniele and Titov, Sergey and Izadi, Maliheh and Bogomolov, Egor and van Deursen, Arie},
howpublished = {arXiv (Cornell University)},
year = {2026},
doi = {10.48550/arxiv.2606.25402},
url = {https://doi.org/10.48550/arxiv.2606.25402}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- API versioning
- software evolution
- large language models
- code generation
- multi-task benchmark
- Python library
- API documentation
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Cipollone, D., Titov, S., Izadi, M., Bogomolov, E., & van Deursen, A. (2026). LibEvoBench: Probing Temporal Knowledge Stratification in Code Generation Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2606.25402
How many times has this paper been cited?
1 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Daniele Cipollone, Sergey Titov, Maliheh Izadi, Egor Bogomolov, Arie van Deursen.
What is the DOI of this paper?
10.48550/arxiv.2606.25402, permanently at https://doi.org/10.48550/arxiv.2606.25402. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.