Preprint · 2026 · Open access
Evaluation Metrics for Safe Reinforcement Learning
arXiv (Cornell University) · Published 14 September 2026 · DOI 10.48550/arxiv.2609.15315
In brief
“Evaluation Metrics for Safe Reinforcement Learning” is a 2026 preprint by Lindsay Spoor and 2 co-authors, published in arXiv (Cornell University). OpenAlex counts 0 citations as of 22 November 2026. It is open access under a CC BY-NC-SA licence: the full text is free to read at arXiv (Cornell University).
- Citations
- 0
- References
- 19
- Authors
- 3
Abstract
Safe reinforcement learning (RL) is commonly formalized as a Constrained Markov Decision Process (CMDP), in which an agent maximizes expected reward while keeping its expected cumulative cost below a specified safety bound. Existing safe RL benchmarks predominantly report whether an algorithm is safe on average, following this expectation-based guarantee. We argue that this convention is insufficient to reliably characterize an algorithm's true safety: it fails to capture how often and how severely the safety bound is violated, whether this holds consistently across tasks and safety bounds, and whether training-time behavior is representative of behavior of the final converged policy. Therefore, we introduce (i) evaluation metrics for safe RL that address each of these concerns and in addition allow for aggregation across tasks and safety bounds. We furthermore define (ii) a safety tier system to systematically categorize and compare algorithms in terms of safety and reliability at both training and for a final policy. Using this framework, we provide (iii) an empirical safety evaluation across multiple safety navigation tasks. Our results show that aggregate metrics, distributional reporting, and task- and safety bound-specific results each reveal information the other metrics cannot. We therefore recommend reporting all three jointly, rather than compressing this information into a single value, as is common practice. We provide SafeRLEval, an open-source evaluation suite to support the reliable characterization of safety in future safe RL research.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Spoor, L., Plaat, A., & Moerland, T. M. (2026). Evaluation Metrics for Safe Reinforcement Learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2609.15315
MLA 9
Spoor, Lindsay, et al. “Evaluation Metrics for Safe Reinforcement Learning.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2609.15315.
Chicago
Spoor, Lindsay, Aske Plaat, and Thomas M. Moerland. 2026. “Evaluation Metrics for Safe Reinforcement Learning.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2609.15315.
Harvard
Spoor, L., Plaat, A. and Moerland, T. M. (2026) ‘Evaluation Metrics for Safe Reinforcement Learning’, arXiv (Cornell University). doi:10.48550/arxiv.2609.15315.
IEEE
L. Spoor, A. Plaat, and T. M. Moerland, “Evaluation Metrics for Safe Reinforcement Learning,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2609.15315.
BibTeX
@misc{spoor2026evaluation,
title = {{Evaluation Metrics for Safe Reinforcement Learning}},
author = {Spoor, Lindsay and Plaat, Aske and Moerland, Thomas M.},
howpublished = {arXiv (Cornell University)},
year = {2026},
doi = {10.48550/arxiv.2609.15315},
url = {https://doi.org/10.48550/arxiv.2609.15315}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- safe reinforcement learning
- constrained Markov decision process
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Spoor, L., Plaat, A., & Moerland, T. M. (2026). Evaluation Metrics for Safe Reinforcement Learning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2609.15315
How many times has this paper been cited?
0 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Lindsay Spoor, Aske Plaat, Thomas M. Moerland.
What is the DOI of this paper?
10.48550/arxiv.2609.15315, permanently at https://doi.org/10.48550/arxiv.2609.15315. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.