Preprint · 2024 · Open access

Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding

Haolin Chen, Yihao Feng, Zuxin Liu, Yao, Weiran, Akshara Prabhakar, Shelby Heinecke, Ricky Ho, Phil Mui, Silvio Savarese, Caiming Xiong, Huan Wang

arXiv (Cornell University) · Published 6 November 2024 · DOI 10.48550/arxiv.2411.04282

Free PDFView at the publisherCite this paper

In brief

“Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding” is a 2024 preprint by Haolin Chen and 10 co-authors, published in arXiv (Cornell University). OpenAlex counts 67 citations as of 15 November 2026. It is open access under a CC BY-SA licence: the full text is free to read at arXiv (Cornell University).

Citations
67
Most cited in
2025 (35)
References
29
Authors
11

Abstract

Large language models (LLMs) have shown impressive capabilities, but still struggle with complex reasoning tasks requiring multiple steps. While prompt-based methods like Chain-of-Thought (CoT) can improve LLM reasoning at inference time, optimizing reasoning capabilities during training remains challenging. We introduce LaTent Reasoning Optimization (LaTRO), a principled framework that formulates reasoning as sampling from a latent distribution and optimizes it via variational approaches. LaTRO enables LLMs to concurrently improve both their reasoning process and ability to evaluate reasoning quality, without requiring external feedback or reward models. We validate LaTRO through experiments on GSM8K and ARC-Challenge datasets using multiple model architectures. On GSM8K, LaTRO improves zero-shot accuracy by an average of 12.5% over base models and 9.6% over supervised fine-tuning across Phi-3.5-mini, Mistral-7B, and Llama-3.1-8B. Our findings suggest that pre-trained LLMs possess latent reasoning capabilities that can be unlocked and enhanced through our proposed optimization approach in a self-improvement manner. The code of LaTRO is available at \url{https://github.com/SalesforceAIResearch/LaTRO}.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Chen, H., Feng, Y., Liu, Z., Weiran, Y., Prabhakar, A., Heinecke, S., Ho, R., Mui, P., Savarese, S., Xiong, C., & Wang, H. (2024). Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2411.04282

MLA 9

Chen, Haolin, et al. “Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding.” arXiv (Cornell University), 2024. https://doi.org/10.48550/arxiv.2411.04282.

Chicago

Chen, Haolin, Yihao Feng, Zuxin Liu, Yao, Weiran, Akshara Prabhakar, Shelby Heinecke, Ricky Ho, et al. 2024. “Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2411.04282.

Harvard

Chen, H. et al. (2024) ‘Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding’, arXiv (Cornell University). doi:10.48550/arxiv.2411.04282.

IEEE

H. Chen et al., “Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding,” arXiv (Cornell University), 2024, doi: 10.48550/arxiv.2411.04282.

BibTeX

@misc{chen2024language,
  title     = {{Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding}},
  author    = {Chen, Haolin and Feng, Yihao and Liu, Zuxin and Weiran, Yao, and Prabhakar, Akshara and Heinecke, Shelby and Ho, Ricky and Mui, Phil and Savarese, Silvio and Xiong, Caiming and Wang, Huan},
  howpublished = {arXiv (Cornell University)},
  year      = {2024},
  doi       = {10.48550/arxiv.2411.04282},
  url       = {https://doi.org/10.48550/arxiv.2411.04282}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling

  • large language models
  • latent reasoning
  • variational inference
  • multi-step reasoning
  • GSM8K
  • ARC-Challenge
  • zero-shot reasoning
  • human development

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Chen, H., Feng, Y., Liu, Z., Weiran, Y., Prabhakar, A., Heinecke, S., Ho, R., Mui, P., Savarese, S., Xiong, C., & Wang, H. (2024). Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2411.04282

How many times has this paper been cited?

67 times, according to OpenAlex on 15 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at arXiv (Cornell University).

Who wrote this paper?

Haolin Chen, Yihao Feng, Zuxin Liu, Yao, Weiran, Akshara Prabhakar, Shelby Heinecke, Ricky Ho, Phil Mui, Silvio Savarese, Caiming Xiong, Huan Wang.

What is the DOI of this paper?

10.48550/arxiv.2411.04282. The permanent link is https://doi.org/10.48550/arxiv.2411.04282.

Bibliographic data from OpenAlex (CC0), last checked 15 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.