Preprint · 2026 · Open access

Evaluation Metrics for Safe Reinforcement Learning

Lindsay Spoor, Aske Plaat, Thomas M. Moerland

arXiv (Cornell University) · Published 14 September 2026 · DOI 10.48550/arxiv.2609.15315

Open PDFView at the publisherCite this paper

In brief

“Evaluation Metrics for Safe Reinforcement Learning” is a 2026 preprint by Lindsay Spoor and 2 co-authors, published in arXiv (Cornell University). OpenAlex counts 0 citations as of 22 November 2026. It is open access under a CC BY-NC-SA licence: the full text is free to read at arXiv (Cornell University).

Citations
0
References
19
Authors
3

Abstract

Safe reinforcement learning (RL) is commonly formalized as a Constrained Markov Decision Process (CMDP), in which an agent maximizes expected reward while keeping its expected cumulative cost below a specified safety bound. Existing safe RL benchmarks predominantly report whether an algorithm is safe on average, following this expectation-based guarantee. We argue that this convention is insufficient to reliably characterize an algorithm's true safety: it fails to capture how often and how severely the safety bound is violated, whether this holds consistently across tasks and safety bounds, and whether training-time behavior is representative of behavior of the final converged policy. Therefore, we introduce (i) evaluation metrics for safe RL that address each of these concerns and in addition allow for aggregation across tasks and safety bounds. We furthermore define (ii) a safety tier system to systematically categorize and compare algorithms in terms of safety and reliability at both training and for a final policy. Using this framework, we provide (iii) an empirical safety evaluation across multiple safety navigation tasks. Our results show that aggregate metrics, distributional reporting, and task- and safety bound-specific results each reveal information the other metrics cannot. We therefore recommend reporting all three jointly, rather than compressing this information into a single value, as is common practice. We provide SafeRLEval, an open-source evaluation suite to support the reliable characterization of safety in future safe RL research.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Spoor, L., Plaat, A., & Moerland, T. M. (2026). Evaluation Metrics for Safe Reinforcement Learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2609.15315

MLA 9

Spoor, Lindsay, et al. “Evaluation Metrics for Safe Reinforcement Learning.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2609.15315.

Chicago

Spoor, Lindsay, Aske Plaat, and Thomas M. Moerland. 2026. “Evaluation Metrics for Safe Reinforcement Learning.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2609.15315.

Harvard

Spoor, L., Plaat, A. and Moerland, T. M. (2026) ‘Evaluation Metrics for Safe Reinforcement Learning’, arXiv (Cornell University). doi:10.48550/arxiv.2609.15315.

IEEE

L. Spoor, A. Plaat, and T. M. Moerland, “Evaluation Metrics for Safe Reinforcement Learning,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2609.15315.

BibTeX

@misc{spoor2026evaluation,
  title     = {{Evaluation Metrics for Safe Reinforcement Learning}},
  author    = {Spoor, Lindsay and Plaat, Aske and Moerland, Thomas M.},
  howpublished = {arXiv (Cornell University)},
  year      = {2026},
  doi       = {10.48550/arxiv.2609.15315},
  url       = {https://doi.org/10.48550/arxiv.2609.15315}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics

  • safe reinforcement learning
  • constrained Markov decision process

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Spoor, L., Plaat, A., & Moerland, T. M. (2026). Evaluation Metrics for Safe Reinforcement Learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2609.15315

How many times has this paper been cited?

0 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Lindsay Spoor, Aske Plaat, Thomas M. Moerland.

What is the DOI of this paper?

10.48550/arxiv.2609.15315, permanently at https://doi.org/10.48550/arxiv.2609.15315. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.