Preprint · 2025 · Open access
Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning
arXiv (Cornell University) · Published 18 December 2025 · DOI 10.48550/arxiv.2512.16917
In brief
“Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning” is a 2025 preprint by Liu, Qihao and 4 co-authors, published in arXiv (Cornell University). OpenAlex counts 6 citations as of 15 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 6
- Most cited in
- 2026 (6)
- References
- 31
- Authors
- 5
Abstract
Large language models (LLMs) with explicit reasoning capabilities excel at mathematical reasoning yet still commit process errors, such as incorrect calculations, brittle logic, and superficially plausible but invalid steps. In this paper, we introduce Generative Adversarial Reasoner, an on-policy joint training framework designed to enhance reasoning by co-evolving an LLM reasoner and an LLM-based discriminator through adversarial reinforcement learning. A compute-efficient review schedule partitions each reasoning chain into logically complete slices of comparable length, and the discriminator evaluates each slice's soundness with concise, structured justifications. Learning couples complementary signals: the LLM reasoner is rewarded for logically consistent steps that yield correct answers, while the discriminator earns rewards for correctly detecting errors or distinguishing traces in the reasoning process. This produces dense, well-calibrated, on-policy step-level rewards that supplement sparse exact-match signals, improving credit assignment, increasing sample efficiency, and enhancing overall reasoning quality of LLMs. Across various mathematical benchmarks, the method delivers consistent gains over strong baselines with standard RL post-training. Specifically, on AIME24, we improve DeepSeek-R1-Distill-Qwen-7B from 54.0 to 61.3 (+7.3) and DeepSeek-R1-Distill-Llama-8B from 43.7 to 53.7 (+10.0). The modular discriminator also enables flexible reward shaping for objectives such as teacher distillation, preference alignment, and mathematical proof-based reasoning.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Qihao, L., Luoxin, Y., Wufei, M., Yu-Cheng, C., & Yuille, A. (2025). Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2512.16917
MLA 9
Qihao, Liu,, et al. “Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2512.16917.
Chicago
Qihao, Liu,, Ye, Luoxin, Ma, Wufei, Chou, Yu-Cheng, and Alan Yuille. 2025. “Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2512.16917.
Harvard
Qihao, L. et al. (2025) ‘Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning’, arXiv (Cornell University). doi:10.48550/arxiv.2512.16917.
IEEE
L. Qihao, Y. Luoxin, M. Wufei, C. Yu-Cheng, and A. Yuille, “Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2512.16917.
BibTeX
@misc{qihao2025generative,
title = {{Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning}},
author = {Qihao, Liu, and Luoxin, Ye, and Wufei, Ma, and Yu-Cheng, Chou, and Yuille, Alan},
howpublished = {arXiv (Cornell University)},
year = {2025},
doi = {10.48550/arxiv.2512.16917},
url = {https://doi.org/10.48550/arxiv.2512.16917}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- large language model reasoning
- adversarial reinforcement learning
- mathematical reasoning
- credit assignment
- reward shaping
- preference alignment
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Qihao, L., Luoxin, Y., Wufei, M., Yu-Cheng, C., & Yuille, A. (2025). Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2512.16917
How many times has this paper been cited?
6 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at arXiv (Cornell University).
Who wrote this paper?
Liu, Qihao, Ye, Luoxin, Ma, Wufei, Chou, Yu-Cheng, Alan Yuille.
What is the DOI of this paper?
10.48550/arxiv.2512.16917. The permanent link is https://doi.org/10.48550/arxiv.2512.16917.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.