Journal article · 2025 · Open access
The architecture of language: Understanding the mechanics behind LLMs
Cambridge Forum on AI Law and Governance, 1 · Published 6 January 2025 · DOI 10.1017/cfl.2024.16
In brief
“The architecture of language: Understanding the mechanics behind LLMs” is a 2025 journal article by Andrea Filippo Ferraris and 3 co-authors, published in Cambridge Forum on AI Law and Governance. OpenAlex counts 51 citations as of 15 November 2026. It is open access under a CC BY licence: the full text is free to read at Cambridge Forum on AI Law and Governance.
- Citations
- 51
- Most cited in
- 2025 (27)
- Field-weighted impact
- 12.23
- References
- 52
- Authors
- 4
Abstract
Large language models (LLMs) have significantly advanced artificial intelligence (AI) and natural language processing (NLP) by excelling in tasks like text generation, machine translation, question answering and sentiment analysis, often rivaling human performance. This paper reviews LLMs’ foundations, advancements and applications, beginning with the transformative transformer architecture, which improved on earlier models like recurrent neural networks and convolutional neural networks through self-attention mechanisms that capture long-range dependencies and contextual relationships. Key innovations such as masked language modeling and causal language modeling underpin leading models like Bidirectional encoder representations from transformers (BERT) and the Generative Pre-trained Transformer (GPT) series. The paper highlights scaling laws, model size increases and advanced training techniques that have driven LLMs’ growth. It also explores methodologies to enhance their precision and adaptability, including parameter-efficient fine-tuning and prompt engineering. Challenges like high computational demands, biases and hallucinations are addressed, with solutions such as retrieval-augmented generation to improve factual accuracy. By discussing LLMs’ strengths, limitations and transformative potential, this paper provides researchers, practitioners and students with a comprehensive understanding. It underscores the importance of ongoing research to improve efficiency, manage ethical concerns and shape the future of AI and language technologies.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Ferraris, A. F., Audrito, D., Di, L., & Poncibò, C. (2025). The architecture of language: Understanding the mechanics behind LLMs. Cambridge Forum on AI Law and Governance, 1. https://doi.org/10.1017/cfl.2024.16
MLA 9
Ferraris, Andrea Filippo, et al. “The architecture of language: Understanding the mechanics behind LLMs.” Cambridge Forum on AI Law and Governance, vol. 1, 2025. https://doi.org/10.1017/cfl.2024.16.
Chicago
Ferraris, Andrea Filippo, Davide Audrito, Luigi Di, and Cristina Poncibò. 2025. “The architecture of language: Understanding the mechanics behind LLMs.” Cambridge Forum on AI Law and Governance 1. https://doi.org/10.1017/cfl.2024.16.
Harvard
Ferraris, A. F. et al. (2025) ‘The architecture of language: Understanding the mechanics behind LLMs’, Cambridge Forum on AI Law and Governance, 1. doi:10.1017/cfl.2024.16.
IEEE
A. F. Ferraris, D. Audrito, L. Di, and C. Poncibò, “The architecture of language: Understanding the mechanics behind LLMs,” Cambridge Forum on AI Law and Governance, vol. 1, 2025, doi: 10.1017/cfl.2024.16.
BibTeX
@article{ferraris2025architecture,
title = {{The architecture of language: Understanding the mechanics behind LLMs}},
author = {Ferraris, Andrea Filippo and Audrito, Davide and Di, Luigi and Poncibò, Cristina},
journal = {Cambridge Forum on AI Law and Governance},
year = {2025},
volume = {1},
doi = {10.1017/cfl.2024.16},
url = {https://doi.org/10.1017/cfl.2024.16}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2024 | 1 | |
| 2025 | 27 | |
| 2026 | 23 |
Its field-weighted citation impact is 12.23: it has been cited 12.23 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- large language models
- transformer models
- self-attention
- masked language modeling
- causal language modeling
- BERT
- GPT
- scaling laws
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Ferraris, A. F., Audrito, D., Di, L., & Poncibò, C. (2025). The architecture of language: Understanding the mechanics behind LLMs. Cambridge Forum on AI Law and Governance, 1. https://doi.org/10.1017/cfl.2024.16
How many times has this paper been cited?
51 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at Cambridge Forum on AI Law and Governance.
Who wrote this paper?
Andrea Filippo Ferraris, Davide Audrito, Luigi Di, Cristina Poncibò.
What is the DOI of this paper?
10.1017/cfl.2024.16. The permanent link is https://doi.org/10.1017/cfl.2024.16.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.