Dissertation · 2025 · Open access

Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion

Wang, Renxi

MBZUAI iRep · Published 1 January 2025 · DOI 10.82419/270

Read the free full textCite this paper

In brief

“Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion” is a 2025 dissertation by Wang, Renxi, published in MBZUAI iRep. OpenAlex counts 0 citations as of 16 November 2026. It is open access: the full text is free to read at MBZUAI iRep.

Citations
0
Authors
1

Abstract

Language agents use large language models (LLMs) to receive environmental observations, make decisions and complete tasks automatically. This thesis investigates and explores the problem of how to train effective language agents that can solve problems with a high success rate. Generally, interfaces that enable language agents to interact with environments can be tools, APIs, or even embodied actions. We start from pure LLMs, a special type of language agent, where the interfaces that they use is a chat box. They use languages to tackle tasks such as answering questions, and interact directly with human users. We first focus on instruction-tuning, which is a critical component for LLM training, and the foundation for agent training. LLMs are shown to follow instructions after train ing with instruction response pairs. We select three types of instructions, representing LLMs’ three critical capabilities: general instruction-following, code generation, and other task-completion capabilities. We find that although these instructions can improve the performance corresponding to their domain, mixing them together may negatively transfer to other domains. Based on instruction-tunning, we extend it into multi-turn inference with tool utilization, and train models with agent-tool interaction trajectories. To mitigate low-resource issues for agent scenarios, we further propose negative-aware agent-tuning, which ap pends positive and negative prefixes and enables LLMs to learn from failed trajectories. Our method is shown to be more effective than simple supervised fine-tuning (SFT) and exploration-based methods, such as Proximal Preference Optimization (PPO) and Direct Preference Optimization (DPO). Finally, we scale up language agents to more available actions that can not fit into LLMs’ context windows. By virtualizing each action into a single token and utilizing three-stage training strategy, we obtain a unified tool retrieval and calling agent through generating tokens. We succeed in integrating 47,000 tools while surpassing previous retrieval-based agent paradigms.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Renxi, W. (2025). Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion. MBZUAI iRep. https://doi.org/10.82419/270

MLA 9

Renxi, Wang,. “Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion.” MBZUAI iRep, 2025. https://doi.org/10.82419/270.

Chicago

Renxi, Wang,. 2025. “Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion.” MBZUAI iRep. https://doi.org/10.82419/270.

Harvard

Renxi, W. (2025) ‘Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion’, MBZUAI iRep. doi:10.82419/270.

IEEE

W. Renxi, “Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion,” MBZUAI iRep, 2025, doi: 10.82419/270.

BibTeX

@phdthesis{renxi2025training,
  title     = {{Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion}},
  author    = {Renxi, Wang,},
  year      = {2025},
  doi       = {10.82419/270},
  url       = {https://doi.org/10.82419/270}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques

  • language agents
  • large language models
  • instruction following
  • instruction tuning
  • tool use
  • negative transfer
  • code generation
  • task completion

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Renxi, W. (2025). Training Effective Language Agents: From Instruction Following to Unified Automatic Task Completion. MBZUAI iRep. https://doi.org/10.82419/270

How many times has this paper been cited?

0 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at MBZUAI iRep.

Who wrote this paper?

Wang, Renxi.

What is the DOI of this paper?

10.82419/270. The permanent link is https://doi.org/10.82419/270.

Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.