Preprint · 2025 · Open access
RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning
arXiv (Cornell University) · Published 20 May 2025 · DOI 10.48550/arxiv.2505.14140
In brief
“RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning” is a 2025 preprint by Qianyue Hao and 3 co-authors, published in arXiv (Cornell University). OpenAlex counts 22 citations as of 15 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 22
- Most cited in
- 2026 (12)
- References
- 36
- Authors
- 4
Abstract
Despite rapid advancements in large language models (LLMs), the token-level autoregressive nature constrains their complex reasoning capabilities. To enhance LLM reasoning, inference-time techniques, including Chain/Tree/Graph-of-Thought(s), successfully improve the performance, as they are fairly cost-effective by guiding reasoning through sophisticated logical structures without modifying LLMs' parameters. However, these manually predefined, task-agnostic frameworks are applied uniformly across diverse tasks, lacking adaptability. To improve this, we propose RL-of-Thoughts (RLoT), where we train a lightweight navigator model with reinforcement learning (RL) to adaptively enhance LLM reasoning at inference time. Specifically, we design five basic logic blocks from the perspective of human cognition. During the reasoning process, the trained RL navigator dynamically selects the suitable logic blocks and combines them into task-specific logical structures according to problem characteristics. Experiments across multiple reasoning benchmarks (AIME, MATH, GPQA, etc.) with multiple LLMs (GPT, Llama, Qwen, and DeepSeek) illustrate that RLoT outperforms established inference-time techniques by up to 13.4%. Remarkably, with less than 3K parameters, our RL navigator is able to make sub-10B LLMs comparable to 100B-scale counterparts. Moreover, the RL navigator demonstrates strong transferability: a model trained on one specific LLM-task pair can effectively generalize to unseen LLMs and tasks. Our code is open-source at https://github.com/tsinghua-fib-lab/RL-LLM-Reasoning for reproducibility.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Hao, Q., Li, S., Yuan, J., & Li, Y. (2025). RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2505.14140
MLA 9
Hao, Qianyue, et al. “RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2505.14140.
Chicago
Hao, Qianyue, Sibo Li, Jian Yuan, and Yong Li. 2025. “RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2505.14140.
Harvard
Hao, Q. et al. (2025) ‘RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning’, arXiv (Cornell University). doi:10.48550/arxiv.2505.14140.
IEEE
Q. Hao, S. Li, J. Yuan, and Y. Li, “RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2505.14140.
BibTeX
@misc{hao2025thoughts,
title = {{RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning}},
author = {Hao, Qianyue and Li, Sibo and Yuan, Jian and Li, Yong},
howpublished = {arXiv (Cornell University)},
year = {2025},
doi = {10.48550/arxiv.2505.14140},
url = {https://doi.org/10.48550/arxiv.2505.14140}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2024 | 1 | |
| 2025 | 9 | |
| 2026 | 12 |
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- LLM reasoning
- chain-of-thought
- tree-of-thought
- graph-of-thought
- logic blocks
- mathematical reasoning
- question answering
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Hao, Q., Li, S., Yuan, J., & Li, Y. (2025). RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2505.14140
How many times has this paper been cited?
22 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at arXiv (Cornell University).
Who wrote this paper?
Qianyue Hao, Sibo Li, Jian Yuan, Yong Li.
What is the DOI of this paper?
10.48550/arxiv.2505.14140. The permanent link is https://doi.org/10.48550/arxiv.2505.14140.
Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.