Journal article · 2024 · Open access
Lost in the Middle: How Language Models Use Long Contexts
Transactions of the Association for Computational Linguistics, 12, 157–173 · Published 1 January 2024 · DOI 10.1162/tacl_a_00638
In brief
“Lost in the Middle: How Language Models Use Long Contexts” is a 2024 journal article by Nelson F. Liu and 6 co-authors, published in Transactions of the Association for Computational Linguistics. OpenAlex counts 3,620 citations as of 15 November 2026. It is open access under a CC BY licence: the full text is free to read at Transactions of the Association for Computational Linguistics.
- Citations
- 3,620
- Most cited in
- 2026 (2,022)
- Field-weighted impact
- 364.00
- References
- 47
- Authors
- 7
Abstract
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638
MLA 9
Liu, Nelson F., et al. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics, vol. 12, 2024, pp. 157–173. https://doi.org/10.1162/tacl_a_00638.
Chicago
Liu, Nelson F., Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics 12: 157–173. https://doi.org/10.1162/tacl_a_00638.
Harvard
Liu, N. F. et al. (2024) ‘Lost in the Middle: How Language Models Use Long Contexts’, Transactions of the Association for Computational Linguistics, 12, pp. 157–173. doi:10.1162/tacl_a_00638.
IEEE
N. F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 157–173, 2024, doi: 10.1162/tacl_a_00638.
BibTeX
@article{liu2024lost,
title = {{Lost in the Middle: How Language Models Use Long Contexts}},
author = {Liu, Nelson F. and Lin, Kevin and Hewitt, John and Paranjape, Ashwin and Bevilacqua, Michele and Petroni, Fabio and Liang, Percy},
journal = {Transactions of the Association for Computational Linguistics},
year = {2024},
volume = {12},
pages = {157--173},
doi = {10.1162/tacl_a_00638},
url = {https://doi.org/10.1162/tacl_a_00638}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2017 | 1 | |
| 2023 | 92 | |
| 2024 | 478 | |
| 2025 | 1,026 | |
| 2026 | 2,022 |
Its field-weighted citation impact is 364.00: it has been cited 364.00 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- long-context language models
- multi-document question answering
- position bias
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638
How many times has this paper been cited?
3,620 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at Transactions of the Association for Computational Linguistics.
Who wrote this paper?
Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang.
What is the DOI of this paper?
10.1162/tacl_a_00638. The permanent link is https://doi.org/10.1162/tacl_a_00638.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.