Journal article · 2026 · Open access

Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control

Rene A. Carmona, Mathieu Laurière

SIAM Review, 68(4), 823–849 · Published 12 November 2026 · DOI 10.1137/25m1782856

Open full textCite this paper

In brief

“Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control” is a 2026 journal article by Rene A. Carmona and Mathieu Laurière, published in SIAM Review. OpenAlex counts 7 citations as of 17 November 2026. It is open access under a CC BY-NC-ND licence: the full text is free to read at SIAM Review.

Citations
7
Most cited in
2026 (6)
Field-weighted impact
5.58
References
47
Authors
2

Abstract

Reinforcement learning (RL) is currently one of the most prominent methods for optimizing dynamical systems, with breakthrough results across various fields. The framework is based on the concept of a Markov decision process (MDP), leading to a discrete-time optimal control problem. In the RL literature, such problems are typically formulated and solved using mixed policies, from which random actions are sampled at each time step. Recently, part of the optimal control community has begun investigating continuous-time versions of RL algorithms, replacing MDPs with continuous-time stochastic processes governed by relaxed controls and asserting a full analogy between the two formulations. In this work, we examine the limitations of this analogy and rigorously establish a connection between the two problems in the case where only the drift term of the continuous-time model is controlled. We prove strong convergence of the RL implementation of mixed strategies as the time discretization mesh tends to zero. We also discuss the technical challenges posed by the possible presence of control in the diffusion component of the state.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Carmona, R. A., & Laurière, M. (2026). Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control. SIAM Review, 68(4), 823–849. https://doi.org/10.1137/25m1782856

MLA 9

Carmona, Rene A., and Mathieu Laurière. “Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control.” SIAM Review, vol. 68, no. 4, 2026, pp. 823–849. https://doi.org/10.1137/25m1782856.

Chicago

Carmona, Rene A. and Mathieu Laurière. 2026. “Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control.” SIAM Review 68 (4): 823–849. https://doi.org/10.1137/25m1782856.

Harvard

Carmona, R. A. and Laurière, M. (2026) ‘Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control’, SIAM Review, 68(4), pp. 823–849. doi:10.1137/25m1782856.

IEEE

R. A. Carmona and M. Laurière, “Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control,” SIAM Review, vol. 68, no. 4, pp. 823–849, 2026, doi: 10.1137/25m1782856.

BibTeX

@article{carmona2026reconciling,
  title     = {{Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control}},
  author    = {Carmona, Rene A. and Laurière, Mathieu},
  journal   = {SIAM Review},
  year      = {2026},
  volume    = {68},
  number    = {4},
  pages     = {823--849},
  doi       = {10.1137/25m1782856},
  url       = {https://doi.org/10.1137/25m1782856}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics

  • reinforcement learning
  • relaxed controls
  • continuous-time stochastic control
  • Markov decision process
  • discrete-time optimal control
  • strong convergence
  • time discretization
  • drift control

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Carmona, R. A., & Laurière, M. (2026). Reconciling Discrete-Time Mixed Policies and Continuous-Time Relaxed Controls in Reinforcement Learning and Stochastic Control. SIAM Review, 68(4), 823–849. https://doi.org/10.1137/25m1782856

How many times has this paper been cited?

7 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at SIAM Review: sign in and open the full text from this page.

Who wrote this paper?

Rene A. Carmona, Mathieu Laurière.

What is the DOI of this paper?

10.1137/25m1782856, permanently at https://doi.org/10.1137/25m1782856. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.