Journal article · 2026 · Open access
Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control
SIAM Review, 68(4), 823–849 · Published 12 November 2026 · DOI 10.1137/25m1782856
In brief
“Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control” is a 2026 journal article by Rene A. Carmona and Mathieu Laurière, published in SIAM Review. OpenAlex counts 7 citations as of 17 November 2026. It is open access under a CC BY-NC-ND licence: the full text is free to read at SIAM Review.
- Citations
- 7
- Most cited in
- 2026 (6)
- Field-weighted impact
- 5.58
- References
- 47
- Authors
- 2
Abstract
Reinforcement learning (RL) is currently one of the most prominent methods for optimizing dynamical systems, with breakthrough results across various fields. The framework is based on the concept of a Markov decision process (MDP), leading to a discrete-time optimal control problem. In the RL literature, such problems are typically formulated and solved using mixed policies, from which random actions are sampled at each time step. Recently, part of the optimal control community has begun investigating continuous-time versions of RL algorithms, replacing MDPs with continuous-time stochastic processes governed by relaxed controls and asserting a full analogy between the two formulations. In this work, we examine the limitations of this analogy and rigorously establish a connection between the two problems in the case where only the drift term of the continuous-time model is controlled. We prove strong convergence of the RL implementation of mixed strategies as the time discretization mesh tends to zero. We also discuss the technical challenges posed by the possible presence of control in the diffusion component of the state.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Carmona, R. A., & Laurière, M. (2026). Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control. SIAM Review, 68(4), 823–849. https://doi.org/10.1137/25m1782856
MLA 9
Carmona, Rene A., and Mathieu Laurière. “Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control.” SIAM Review, vol. 68, no. 4, 2026, pp. 823–849. https://doi.org/10.1137/25m1782856.
Chicago
Carmona, Rene A. and Mathieu Laurière. 2026. “Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control.” SIAM Review 68 (4): 823–849. https://doi.org/10.1137/25m1782856.
Harvard
Carmona, R. A. and Laurière, M. (2026) ‘Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control’, SIAM Review, 68(4), pp. 823–849. doi:10.1137/25m1782856.
IEEE
R. A. Carmona and M. Laurière, “Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control,” SIAM Review, vol. 68, no. 4, pp. 823–849, 2026, doi: 10.1137/25m1782856.
BibTeX
@article{carmona2026reconciling,
title = {{Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control}},
author = {Carmona, Rene A. and Laurière, Mathieu},
journal = {SIAM Review},
year = {2026},
volume = {68},
number = {4},
pages = {823--849},
doi = {10.1137/25m1782856},
url = {https://doi.org/10.1137/25m1782856}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- reinforcement learning
- relaxed controls
- continuous-time stochastic control
- Markov decision process
- discrete-time optimal control
- strong convergence
- time discretization
- drift control
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Carmona, R. A., & Laurière, M. (2026). Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control. SIAM Review, 68(4), 823–849. https://doi.org/10.1137/25m1782856
How many times has this paper been cited?
7 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at SIAM Review: sign in and open the full text from this page.
Who wrote this paper?
Rene A. Carmona, Mathieu Laurière.
What is the DOI of this paper?
10.1137/25m1782856, permanently at https://doi.org/10.1137/25m1782856. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.