Journal article · 2026 · Open access
Can LLMs Generate User Stories and Assess Their Quality?
IEEE Transactions on Software Engineering, 52(5), 1773–1790 · Published 11 March 2026 · DOI 10.1109/tse.2026.3670612
In brief
“Can LLMs Generate User Stories and Assess Their Quality?” is a 2026 journal article by Giovanni Quattrocchi and 3 co-authors, published in IEEE Transactions on Software Engineering. OpenAlex counts 20 citations as of 18 November 2026, which puts it in the top 1% of works published in 2026 by citations. It is open access under a CC BY licence: the full text is free to read at IEEE Transactions on Software Engineering.
- Citations
- 20
- Most cited in
- 2026 (20)
- Field-weighted impact
- 53.42
- References
- 63
- Authors
- 4
Abstract
Requirements elicitation is still one of the most challenging activities of the requirements engineering process due to the difficulty requirements analysts face in understanding and translating complex needs into concrete requirements that directly impact the quality of the software to be developed. Although automated tools allow for assessing the syntactic quality of requirements, evaluating semantic metrics (e.g., language clarity, internal consistency) remains a manual and time-consuming activity. This paper explores how LLMs can help automate requirements elicitation within agile frameworks, where requirements are defined as user stories. We used 10 state-of-the-art LLMs to investigate their ability to generate user stories automatically by emulating customer interviews. We evaluated the quality of user stories generated by LLMs, comparing it with the quality of user stories generated by humans (domain experts and students). We also explored whether and how LLMs can be used to automatically evaluate the semantic quality of user stories. Our results indicate that LLMs can generate user stories similar to humans in terms of coverage and stylistic quality, but exhibit lower diversity and creativity. Although LLM-generated user stories are generally comparable in quality to those created by humans, they tend to meet the acceptance quality criteria less frequently, regardless of the scale of the LLM model. Finally, LLMs can reliably assess the semantic quality of user stories when provided with clear evaluation criteria and have the potential to reduce human effort in large-scale assessments.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Quattrocchi, G., Pasquale, L., Spoletini, P., & Baresi, L. (2026). Can LLMs Generate User Stories and Assess Their Quality? IEEE Transactions on Software Engineering, 52(5), 1773–1790. https://doi.org/10.1109/tse.2026.3670612
MLA 9
Quattrocchi, Giovanni, et al. “Can LLMs Generate User Stories and Assess Their Quality?” IEEE Transactions on Software Engineering, vol. 52, no. 5, 2026, pp. 1773–1790. https://doi.org/10.1109/tse.2026.3670612.
Chicago
Quattrocchi, Giovanni, Liliana Pasquale, Paola Spoletini, and Luciano Baresi. 2026. “Can LLMs Generate User Stories and Assess Their Quality?” IEEE Transactions on Software Engineering 52 (5): 1773–1790. https://doi.org/10.1109/tse.2026.3670612.
Harvard
Quattrocchi, G. et al. (2026) ‘Can LLMs Generate User Stories and Assess Their Quality?’, IEEE Transactions on Software Engineering, 52(5), pp. 1773–1790. doi:10.1109/tse.2026.3670612.
IEEE
G. Quattrocchi, L. Pasquale, P. Spoletini, and L. Baresi, “Can LLMs Generate User Stories and Assess Their Quality?,” IEEE Transactions on Software Engineering, vol. 52, no. 5, pp. 1773–1790, 2026, doi: 10.1109/tse.2026.3670612.
BibTeX
@article{quattrocchi2026llms,
title = {{Can LLMs Generate User Stories and Assess Their Quality?}},
author = {Quattrocchi, Giovanni and Pasquale, Liliana and Spoletini, Paola and Baresi, Luciano},
journal = {IEEE Transactions on Software Engineering},
year = {2026},
volume = {52},
number = {5},
pages = {1773--1790},
doi = {10.1109/tse.2026.3670612},
url = {https://doi.org/10.1109/tse.2026.3670612}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- large language models
- user stories
- requirements analysis
- requirements engineering
- customer interviews
- agile software development
- acceptance criteria
- human vs LLM comparison
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Quattrocchi, G., Pasquale, L., Spoletini, P., & Baresi, L. (2026). Can LLMs Generate User Stories and Assess Their Quality? IEEE Transactions on Software Engineering, 52(5), 1773–1790. https://doi.org/10.1109/tse.2026.3670612
How many times has this paper been cited?
20 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at IEEE Transactions on Software Engineering: sign in and open the full text from this page.
Who wrote this paper?
Giovanni Quattrocchi, Liliana Pasquale, Paola Spoletini, Luciano Baresi.
What is the DOI of this paper?
10.1109/tse.2026.3670612, permanently at https://doi.org/10.1109/tse.2026.3670612. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.