Journal article · 2025 · Open access

The architecture of language: Understanding the mechanics behind LLMs

Andrea Filippo Ferraris, Davide Audrito, Luigi Di, Cristina Poncibò

Cambridge Forum on AI Law and Governance, 1 · Published 6 January 2025 · DOI 10.1017/cfl.2024.16

Free PDFView at the publisherCite this paper

In brief

“The architecture of language: Understanding the mechanics behind LLMs” is a 2025 journal article by Andrea Filippo Ferraris and 3 co-authors, published in Cambridge Forum on AI Law and Governance. OpenAlex counts 51 citations as of 15 November 2026. It is open access under a CC BY licence: the full text is free to read at Cambridge Forum on AI Law and Governance.

Citations
51
Most cited in
2025 (27)
Field-weighted impact
12.23
References
52
Authors
4

Abstract

Large language models (LLMs) have significantly advanced artificial intelligence (AI) and natural language processing (NLP) by excelling in tasks like text generation, machine translation, question answering and sentiment analysis, often rivaling human performance. This paper reviews LLMs’ foundations, advancements and applications, beginning with the transformative transformer architecture, which improved on earlier models like recurrent neural networks and convolutional neural networks through self-attention mechanisms that capture long-range dependencies and contextual relationships. Key innovations such as masked language modeling and causal language modeling underpin leading models like Bidirectional encoder representations from transformers (BERT) and the Generative Pre-trained Transformer (GPT) series. The paper highlights scaling laws, model size increases and advanced training techniques that have driven LLMs’ growth. It also explores methodologies to enhance their precision and adaptability, including parameter-efficient fine-tuning and prompt engineering. Challenges like high computational demands, biases and hallucinations are addressed, with solutions such as retrieval-augmented generation to improve factual accuracy. By discussing LLMs’ strengths, limitations and transformative potential, this paper provides researchers, practitioners and students with a comprehensive understanding. It underscores the importance of ongoing research to improve efficiency, manage ethical concerns and shape the future of AI and language technologies.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Ferraris, A. F., Audrito, D., Di, L., & Poncibò, C. (2025). The architecture of language: Understanding the mechanics behind LLMs. Cambridge Forum on AI Law and Governance, 1. https://doi.org/10.1017/cfl.2024.16

MLA 9

Ferraris, Andrea Filippo, et al. “The architecture of language: Understanding the mechanics behind LLMs.” Cambridge Forum on AI Law and Governance, vol. 1, 2025. https://doi.org/10.1017/cfl.2024.16.

Chicago

Ferraris, Andrea Filippo, Davide Audrito, Luigi Di, and Cristina Poncibò. 2025. “The architecture of language: Understanding the mechanics behind LLMs.” Cambridge Forum on AI Law and Governance 1. https://doi.org/10.1017/cfl.2024.16.

Harvard

Ferraris, A. F. et al. (2025) ‘The architecture of language: Understanding the mechanics behind LLMs’, Cambridge Forum on AI Law and Governance, 1. doi:10.1017/cfl.2024.16.

IEEE

A. F. Ferraris, D. Audrito, L. Di, and C. Poncibò, “The architecture of language: Understanding the mechanics behind LLMs,” Cambridge Forum on AI Law and Governance, vol. 1, 2025, doi: 10.1017/cfl.2024.16.

BibTeX

@article{ferraris2025architecture,
  title     = {{The architecture of language: Understanding the mechanics behind LLMs}},
  author    = {Ferraris, Andrea Filippo and Audrito, Davide and Di, Luigi and Poncibò, Cristina},
  journal   = {Cambridge Forum on AI Law and Governance},
  year      = {2025},
  volume    = {1},
  doi       = {10.1017/cfl.2024.16},
  url       = {https://doi.org/10.1017/cfl.2024.16}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202411
20252727
20262323

Its field-weighted citation impact is 12.23: it has been cited 12.23 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling

  • large language models
  • transformer models
  • self-attention
  • masked language modeling
  • causal language modeling
  • BERT
  • GPT
  • scaling laws

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Ferraris, A. F., Audrito, D., Di, L., & Poncibò, C. (2025). The architecture of language: Understanding the mechanics behind LLMs. Cambridge Forum on AI Law and Governance, 1. https://doi.org/10.1017/cfl.2024.16

How many times has this paper been cited?

51 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at Cambridge Forum on AI Law and Governance.

Who wrote this paper?

Andrea Filippo Ferraris, Davide Audrito, Luigi Di, Cristina Poncibò.

What is the DOI of this paper?

10.1017/cfl.2024.16. The permanent link is https://doi.org/10.1017/cfl.2024.16.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.