Dissertation · 2025 · Open access
Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion
MBZUAI iRep · Published 1 January 2025 · DOI 10.82419/270
In brief
“Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion” is a 2025 dissertation by Wang, Renxi, published in MBZUAI iRep. OpenAlex counts 0 citations as of 16 November 2026. It is open access: the full text is free to read at MBZUAI iRep.
- Citations
- 0
- Authors
- 1
Abstract
Language agents use large language models (LLMs) to receive environmental observations, make decisions and complete tasks automatically. This thesis investigates and explores the problem of how to train effective language agents that can solve problems with a high success rate. Generally, interfaces that enable language agents to interact with environments can be tools, APIs, or even embodied actions. We start from pure LLMs, a special type of language agent, where the interfaces that they use is a chat box. They use languages to tackle tasks such as answering questions, and interact directly with human users. We first focus on instruction-tuning, which is a critical component for LLM training, and the foundation for agent training. LLMs are shown to follow instructions after train ing with instruction response pairs. We select three types of instructions, representing LLMs’ three critical capabilities: general instruction-following, code generation, and other task-completion capabilities. We find that although these instructions can improve the performance corresponding to their domain, mixing them together may negatively transfer to other domains. Based on instruction-tunning, we extend it into multi-turn inference with tool utilization, and train models with agent-tool interaction trajectories. To mitigate low-resource issues for agent scenarios, we further propose negative-aware agent-tuning, which ap pends positive and negative prefixes and enables LLMs to learn from failed trajectories. Our method is shown to be more effective than simple supervised fine-tuning (SFT) and exploration-based methods, such as Proximal Preference Optimization (PPO) and Direct Preference Optimization (DPO). Finally, we scale up language agents to more available actions that can not fit into LLMs’ context windows. By virtualizing each action into a single token and utilizing three-stage training strategy, we obtain a unified tool retrieval and calling agent through generating tokens. We succeed in integrating 47,000 tools while surpassing previous retrieval-based agent paradigms.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Renxi, W. (2025). Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion. MBZUAI iRep. https://doi.org/10.82419/270
MLA 9
Renxi, Wang,. “Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion.” MBZUAI iRep, 2025. https://doi.org/10.82419/270.
Chicago
Renxi, Wang,. 2025. “Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion.” MBZUAI iRep. https://doi.org/10.82419/270.
Harvard
Renxi, W. (2025) ‘Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion’, MBZUAI iRep. doi:10.82419/270.
IEEE
W. Renxi, “Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion,” MBZUAI iRep, 2025, doi: 10.82419/270.
BibTeX
@phdthesis{renxi2025training,
title = {{Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion}},
author = {Renxi, Wang,},
year = {2025},
doi = {10.82419/270},
url = {https://doi.org/10.82419/270}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- language agents
- large language models
- instruction following
- instruction tuning
- tool use
- negative transfer
- code generation
- task completion
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Renxi, W. (2025). Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion. MBZUAI iRep. https://doi.org/10.82419/270
How many times has this paper been cited?
0 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at MBZUAI iRep.
Who wrote this paper?
Wang, Renxi.
What is the DOI of this paper?
10.82419/270. The permanent link is https://doi.org/10.82419/270.
Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.