Journal article · 2026 · Open access

Efficient and accurate steering of large language models through attention-guided feature learning

Parmida Davarmanesh, Ashia Wilson, Adityanarayanan Radhakrishnan

Proceedings of the National Academy of Sciences, 123(42), e2609978123 · Published 16 October 2026 · DOI 10.1073/pnas.2609978123

Open full textCite this paper

In brief

“Efficient and accurate steering of large language models through attention-guided feature learning” is a 2026 journal article by Parmida Davarmanesh and 2 co-authors, published in Proceedings of the National Academy of Sciences. OpenAlex counts 7 citations as of 17 November 2026, which puts it in the top 2% of works published in 2026 by citations. It is open access under a CC BY-NC-ND licence: the full text is free to read at Proceedings of the National Academy of Sciences.

Citations
7
Most cited in
2026 (7)
Field-weighted impact
9.77
References
53
Authors
3

Abstract

Activation steering, or direct manipulation of internal activations to guide large language model (LLM) responses toward specific semantic concepts, is emerging as a promising avenue for both understanding how semantic concepts are stored within LLMs and advancing LLM capabilities. Yet, existing steering methods are remarkably brittle, with seemingly nonsteerable concepts becoming completely steerable based on subtle algorithmic choices in how concept-related features are extracted. In this work, we introduce an attention-guided framework for activation steering that overcomes three core challenges associated with steering: 1) automatic selection of relevant token embeddings for extracting concept-related features; 2) accounting for heterogeneity of concept-related features across LLM activations; and 3) identification of layers most relevant for steering. Across a steering benchmark of 512 semantic concepts, our framework substantially improved steering over previous state-of-the-art (nearly doubling the number of successfully steered concepts) across model architectures and sizes (up to 70 billion parameter models). Furthermore, we use our framework to shed light on the distribution of concept-specific features across LLM layers. Overall, our framework opens further avenues for developing efficient, highly scalable fine-tuning algorithms for industry-scale LLMs.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Davarmanesh, P., Wilson, A., & Radhakrishnan, A. (2026). Efficient and accurate steering of large language models through attention-guided feature learning. Proceedings of the National Academy of Sciences, 123(42), e2609978123. https://doi.org/10.1073/pnas.2609978123

MLA 9

Davarmanesh, Parmida, et al. “Efficient and accurate steering of large language models through attention-guided feature learning.” Proceedings of the National Academy of Sciences, vol. 123, no. 42, 2026, p. e2609978123. https://doi.org/10.1073/pnas.2609978123.

Chicago

Davarmanesh, Parmida, Ashia Wilson, and Adityanarayanan Radhakrishnan. 2026. “Efficient and accurate steering of large language models through attention-guided feature learning.” Proceedings of the National Academy of Sciences 123 (42): e2609978123. https://doi.org/10.1073/pnas.2609978123.

Harvard

Davarmanesh, P., Wilson, A. and Radhakrishnan, A. (2026) ‘Efficient and accurate steering of large language models through attention-guided feature learning’, Proceedings of the National Academy of Sciences, 123(42), p. e2609978123. doi:10.1073/pnas.2609978123.

IEEE

P. Davarmanesh, A. Wilson, and A. Radhakrishnan, “Efficient and accurate steering of large language models through attention-guided feature learning,” Proceedings of the National Academy of Sciences, vol. 123, no. 42, p. e2609978123, 2026, doi: 10.1073/pnas.2609978123.

BibTeX

@article{davarmanesh2026efficient,
  title     = {{Efficient and accurate steering of large language models through attention-guided feature learning}},
  author    = {Davarmanesh, Parmida and Wilson, Ashia and Radhakrishnan, Adityanarayanan},
  journal   = {Proceedings of the National Academy of Sciences},
  year      = {2026},
  volume    = {123},
  number    = {42},
  pages     = {e2609978123},
  doi       = {10.1073/pnas.2609978123},
  url       = {https://doi.org/10.1073/pnas.2609978123}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling

  • activation steering
  • large language models
  • token embeddings
  • layer selection
  • semantic concepts
  • LLM interpretability

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Davarmanesh, P., Wilson, A., & Radhakrishnan, A. (2026). Efficient and accurate steering of large language models through attention-guided feature learning. Proceedings of the National Academy of Sciences, 123(42), e2609978123. https://doi.org/10.1073/pnas.2609978123

How many times has this paper been cited?

7 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Proceedings of the National Academy of Sciences: sign in and open the full text from this page.

Who wrote this paper?

Parmida Davarmanesh, Ashia Wilson, Adityanarayanan Radhakrishnan.

What is the DOI of this paper?

10.1073/pnas.2609978123, permanently at https://doi.org/10.1073/pnas.2609978123. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.