Preprint · 2023 · Open access

SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization

Jaafar Mhamed, Shangding Gu

arXiv (Cornell University) · Published 1 November 2023 · DOI 10.48550/arxiv.2311.00880

Open PDFView at the publisherCite this paper

In brief

“SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization” is a 2023 preprint by Jaafar Mhamed and Shangding Gu, published in arXiv (Cornell University). OpenAlex counts 1 citations as of 22 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
1
Most cited in
2025 (1)
References
28
Authors
2

Abstract

Incorporating safety is an essential prerequisite for broadening the practical applications of reinforcement learning in real-world scenarios. To tackle this challenge, Constrained Markov Decision Processes (CMDPs) are leveraged, which introduce a distinct cost function representing safety violations. In CMDPs' settings, Lagrangian relaxation technique has been employed in previous algorithms to convert constrained optimization problems into unconstrained dual problems. However, these algorithms may inaccurately predict unsafe behavior, resulting in instability while learning the Lagrange multiplier. This study introduces a novel safe reinforcement learning algorithm, Safety Critic Policy Optimization (SCPO). In this study, we define the safety critic, a mechanism that nullifies rewards obtained through violating safety constraints. Furthermore, our theoretical analysis indicates that the proposed algorithm can automatically balance the trade-off between adhering to safety constraints and maximizing rewards. The effectiveness of the SCPO algorithm is empirically validated by benchmarking it against strong baselines.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Mhamed, J., & Gu, S. (2023). SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2311.00880

MLA 9

Mhamed, Jaafar, and Shangding Gu. “SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization.” arXiv (Cornell University), 2023. https://doi.org/10.48550/arxiv.2311.00880.

Chicago

Mhamed, Jaafar and Shangding Gu. 2023. “SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2311.00880.

Harvard

Mhamed, J. and Gu, S. (2023) ‘SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization’, arXiv (Cornell University). doi:10.48550/arxiv.2311.00880.

IEEE

J. Mhamed and S. Gu, “SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization,” arXiv (Cornell University), 2023, doi: 10.48550/arxiv.2311.00880.

BibTeX

@misc{mhamed2023scpo,
  title     = {{SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization}},
  author    = {Mhamed, Jaafar and Gu, Shangding},
  howpublished = {arXiv (Cornell University)},
  year      = {2023},
  doi       = {10.48550/arxiv.2311.00880},
  url       = {https://doi.org/10.48550/arxiv.2311.00880}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics

  • safe reinforcement learning
  • constrained Markov decision process
  • Lagrangian relaxation
  • safety constraints
  • policy optimization

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Mhamed, J., & Gu, S. (2023). SCPO: Safe Reinforcement Learning with Safety Critic Policy Optimization. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2311.00880

How many times has this paper been cited?

1 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Jaafar Mhamed, Shangding Gu.

What is the DOI of this paper?

10.48550/arxiv.2311.00880, permanently at https://doi.org/10.48550/arxiv.2311.00880. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.