Review article · 2026 · Open access
A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(10), 11382–11401 · Published 28 April 2026 · DOI 10.1109/tpami.2026.3688650
In brief
“A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents” is a 2026 review article by Hang Su and 6 co-authors, published in IEEE Transactions on Pattern Analysis and Machine Intelligence. OpenAlex counts 75 citations as of 18 November 2026, which puts it in the top 4% of works published in 2026 by citations. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 75
- Most cited in
- 2026 (69)
- Field-weighted impact
- 27.69
- References
- 148
- Authors
- 7
Abstract
Recent advances in large language models (LLMs) have catalyzed the rise of autonomous AI agents capable of perceiving, reasoning, and acting in dynamic, open-ended environments. These large-model agents mark a paradigm shift from static inference systems to interactive, memory-augmented entities. While these capabilities significantly expand the functional scope of AI, they also introduce qualitatively novel security risks-such as memory poisoning, tool misuse, reward hacking, and emergent misalignment-that extend beyond the threat models of conventional systems or standalone LLMs. In this survey, we first examine the structural foundations and key capabilities that underpin increasing levels of agent autonomy, including long-term memory retention, modular tool use, recursive planning, and reflective reasoning. We then analyze the corresponding security vulnerabilities across the agent stack, identifying failure modes such as deferred decision hazards, irreversible tool chains, and deceptive behaviors arising from internal state drift or value misalignment. These risks are traced to architectural fragilities that emerge across perception, cognition, memory, and action modules. To address these challenges, we systematically review recent defense strategies deployed at different autonomy layers, including input sanitization, memory lifecycle control, constrained decision-making, structured tool invocation, and introspective reflection. While these techniques provide partial mitigation, most operate in isolation and lack the integrated coherence required to manage emergent, temporally extended, and cross-module threats. Motivated by these limitations, we introduce the Reflective Risk-Aware Agent Architecture (R2A2)-a unified cognitive framework grounded in Constrained Markov Decision Processes (CMDPs), which incorporates risk-aware world modeling, meta-policy adaptation, and joint reward-risk optimization to enable principled, proactive safety across the agent's decision-making loop. This survey provides a structured understanding of how autonomy reshapes the security landscape of intelligent systems and offers a blueprint for embedding safety as a core design principle in next-generation AI agents.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Su, H., Luo, J., Liu, C., Yang, X. H., Zhang, Y., Dong, Y., & Zhu, J. J. (2026). A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(10), 11382–11401. https://doi.org/10.1109/tpami.2026.3688650
MLA 9
Su, Hang, et al. “A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents.” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 48, no. 10, 2026, pp. 11382–11401. https://doi.org/10.1109/tpami.2026.3688650.
Chicago
Su, Hang, Jun Luo, Chang Liu, Xiao Hui Yang, Yichi Zhang, Yinpeng Dong, and Jun Jie Zhu. 2026. “A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents.” IEEE Transactions on Pattern Analysis and Machine Intelligence 48 (10): 11382–11401. https://doi.org/10.1109/tpami.2026.3688650.
Harvard
Su, H. et al. (2026) ‘A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents’, IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(10), pp. 11382–11401. doi:10.1109/tpami.2026.3688650.
IEEE
H. Su et al., “A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 48, no. 10, pp. 11382–11401, 2026, doi: 10.1109/tpami.2026.3688650.
BibTeX
@article{su2026survey,
title = {{A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents}},
author = {Su, Hang and Luo, Jun and Liu, Chang and Yang, Xiao Hui and Zhang, Yichi and Dong, Yinpeng and Zhu, Jun Jie},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {10},
pages = {11382--11401},
doi = {10.1109/tpami.2026.3688650},
url = {https://doi.org/10.1109/tpami.2026.3688650}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- autonomous AI agents
- large language models
- AI security risks
- memory poisoning
- tool misuse
- reward hacking
- emergent misalignment
- agent autonomy
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Su, H., Luo, J., Liu, C., Yang, X. H., Zhang, Y., Dong, Y., & Zhu, J. J. (2026). A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(10), 11382–11401. https://doi.org/10.1109/tpami.2026.3688650
How many times has this paper been cited?
75 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Hang Su, Jun Luo, Chang Liu, Xiao Hui Yang, Yichi Zhang, Yinpeng Dong, Jun Jie Zhu.
What is the DOI of this paper?
10.1109/tpami.2026.3688650, permanently at https://doi.org/10.1109/tpami.2026.3688650. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.