Conference paper · 2023 · Open access

A Multiplicative Value Function for Safe and Efficient Reinforcement Learning

Nick Bührer, Zhejun Zhang, Alexander Liniger, Fisher Yu, Luc Van Gool

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 5582–5589 · Published 1 October 2023 · DOI 10.1109/iros55552.2023.10342288

Open PDFView at the publisherCite this paper

In brief

“A Multiplicative Value Function for Safe and Efficient Reinforcement Learning” is a 2023 conference paper by Nick Bührer and 4 co-authors, published in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). OpenAlex counts 5 citations as of 22 November 2026. It is open access: the full text is free to read at Repository for Publications and Research Data (ETH Zurich).

Citations
5
Most cited in
2025 (2)
Field-weighted impact
0.31
References
45
Authors
5

Abstract

An emerging field of sequential decision problems is safe Reinforcement Learning (RL), where the objective is to maximize the reward while obeying safety constraints. Being able to handle constraints is essential for deploying RL agents in real-world environments, where constraint violations can harm the agent and the environment. To this end, we propose a safe model-free RL algorithm with a novel multiplicative value function consisting of a safety critic and a reward critic. The safety critic predicts the probability of constraint violation and discounts the reward critic that only estimates constraint-free returns. By splitting responsibilities, we facilitate the learning task leading to increased sample efficiency. We integrate our approach into two popular RL algorithms, Proximal Policy Optimization and Soft Actor-Critic, and evaluate our method in four safety-focused environments, including classical RL benchmarks augmented with safety constraints and robot navigation tasks with images and raw Lidar scans as observations. Finally, we make the zero-shot sim-to-real transfer where a differential drive robot has to navigate through a cluttered room. Our code can be found at https://github.com/nikeke19/Safe-Mult-RL.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Bührer, N., Zhang, Z., Liniger, A., Yu, F., & Van Gool, L. (2023). A Multiplicative Value Function for Safe and Efficient Reinforcement Learning. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 5582–5589. https://doi.org/10.1109/iros55552.2023.10342288

MLA 9

Bührer, Nick, et al. “A Multiplicative Value Function for Safe and Efficient Reinforcement Learning.” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023, pp. 5582–5589. https://doi.org/10.1109/iros55552.2023.10342288.

Chicago

Bührer, Nick, Zhejun Zhang, Alexander Liniger, Fisher Yu, and Luc Van Gool. 2023. “A Multiplicative Value Function for Safe and Efficient Reinforcement Learning.” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS): 5582–5589. https://doi.org/10.1109/iros55552.2023.10342288.

Harvard

Bührer, N. et al. (2023) ‘A Multiplicative Value Function for Safe and Efficient Reinforcement Learning’, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5582–5589. doi:10.1109/iros55552.2023.10342288.

IEEE

N. Bührer, Z. Zhang, A. Liniger, F. Yu, and L. Van Gool, “A Multiplicative Value Function for Safe and Efficient Reinforcement Learning,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5582–5589, 2023, doi: 10.1109/iros55552.2023.10342288.

BibTeX

@inproceedings{buhrer2023multiplicative,
  title     = {{A Multiplicative Value Function for Safe and Efficient Reinforcement Learning}},
  author    = {Bührer, Nick and Zhang, Zhejun and Liniger, Alexander and Yu, Fisher and Van Gool, Luc},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year      = {2023},
  pages     = {5582--5589},
  doi       = {10.1109/iros55552.2023.10342288},
  url       = {https://doi.org/10.1109/iros55552.2023.10342288}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202411
202522
202622

Its field-weighted citation impact is 0.31: it has been cited 0.31 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics

  • safe reinforcement learning
  • safety constraints
  • constraint violation probability
  • sample efficiency
  • proximal policy optimization
  • soft actor-critic
  • sim-to-real transfer
  • robot navigation

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Bührer, N., Zhang, Z., Liniger, A., Yu, F., & Van Gool, L. (2023). A Multiplicative Value Function for Safe and Efficient Reinforcement Learning. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 5582–5589. https://doi.org/10.1109/iros55552.2023.10342288

How many times has this paper been cited?

5 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Repository for Publications and Research Data (ETH Zurich): sign in and open the PDF from this page.

Who wrote this paper?

Nick Bührer, Zhejun Zhang, Alexander Liniger, Fisher Yu, Luc Van Gool.

What is the DOI of this paper?

10.1109/iros55552.2023.10342288, permanently at https://doi.org/10.1109/iros55552.2023.10342288. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.