Journal article · 2024 · Open access

Lost in the Middle: How Language Models Use Long Contexts

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang

Transactions of the Association for Computational Linguistics, 12, 157–173 · Published 1 January 2024 · DOI 10.1162/tacl_a_00638

Free PDFView at the publisherCite this paper

In brief

“Lost in the Middle: How Language Models Use Long Contexts” is a 2024 journal article by Nelson F. Liu and 6 co-authors, published in Transactions of the Association for Computational Linguistics. OpenAlex counts 3,620 citations as of 15 November 2026. It is open access under a CC BY licence: the full text is free to read at Transactions of the Association for Computational Linguistics.

Citations
3,620
Most cited in
2026 (2,022)
Field-weighted impact
364.00
References
47
Authors
7

Abstract

While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638

MLA 9

Liu, Nelson F., et al. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics, vol. 12, 2024, pp. 157–173. https://doi.org/10.1162/tacl_a_00638.

Chicago

Liu, Nelson F., Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics 12: 157–173. https://doi.org/10.1162/tacl_a_00638.

Harvard

Liu, N. F. et al. (2024) ‘Lost in the Middle: How Language Models Use Long Contexts’, Transactions of the Association for Computational Linguistics, 12, pp. 157–173. doi:10.1162/tacl_a_00638.

IEEE

N. F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 157–173, 2024, doi: 10.1162/tacl_a_00638.

BibTeX

@article{liu2024lost,
  title     = {{Lost in the Middle: How Language Models Use Long Contexts}},
  author    = {Liu, Nelson F. and Lin, Kevin and Hewitt, John and Paranjape, Ashwin and Bevilacqua, Michele and Petroni, Fabio and Liang, Percy},
  journal   = {Transactions of the Association for Computational Linguistics},
  year      = {2024},
  volume    = {12},
  pages     = {157--173},
  doi       = {10.1162/tacl_a_00638},
  url       = {https://doi.org/10.1162/tacl_a_00638}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
201711
20239292
2024478478
20251,0261026
20262,0222022

Its field-weighted citation impact is 364.00: it has been cited 364.00 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling

  • long-context language models
  • multi-document question answering
  • position bias

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638

How many times has this paper been cited?

3,620 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at Transactions of the Association for Computational Linguistics.

Who wrote this paper?

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang.

What is the DOI of this paper?

10.1162/tacl_a_00638. The permanent link is https://doi.org/10.1162/tacl_a_00638.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.