Conference paper · 2021 · Open access
Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3506–3511 · Published 27 September 2021 · DOI 10.1109/iros51168.2021.9636480
In brief
“Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning” is a 2021 conference paper by Kasra Rezaee and 2 co-authors, published in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). OpenAlex counts 32 citations as of 22 November 2026, which puts it in the top 8% of works published in 2021 by citations. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 32
- Most cited in
- 2024 (12)
- Field-weighted impact
- 1.59
- References
- 24
- Authors
- 3
Abstract
Motion planning under uncertainty is one of the main challenges in developing autonomous driving vehicles. In this work, we focus on the uncertainty in sensing and perception, resulted from a limited field of view, occlusions, and sensing range. This problem is often tackled by considering hypothetical hidden objects in occluded areas or beyond the sensing range to guarantee passive safety. However, this may result in conservative planning and expensive computation, particularly when numerous hypothetical objects need to be considered. We propose a reinforcement learning (RL) based solution to manage uncertainty by optimizing for the worst case outcome. This approach is in contrast to traditional RL, where the agents try to maximize the average expected reward. The proposed approach is built on top of the Distributional RL with its policy optimization maximizing the stochastic outcomes’ lower bound. This modification can be applied to a range of RL algorithms. As a proof-of-concept, the approach is applied to two different RL algorithms, Soft Actor-Critic and DQN. The approach is evaluated against two challenging scenarios of pedestrians crossing with occlusion and curved roads with a limited field of view. The algorithm is trained and evaluated using the SUMO traffic simulator. The proposed approach yields much better motion planning behavior compared to conventional RL algorithms and behaves comparably to humans driving style.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Rezaee, K., Yadmellat, P., & Chamorro, S. (2021). Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3506–3511. https://doi.org/10.1109/iros51168.2021.9636480
MLA 9
Rezaee, Kasra, et al. “Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning.” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021, pp. 3506–3511. https://doi.org/10.1109/iros51168.2021.9636480.
Chicago
Rezaee, Kasra, Peyman Yadmellat, and Simon Chamorro. 2021. “Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning.” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS): 3506–3511. https://doi.org/10.1109/iros51168.2021.9636480.
Harvard
Rezaee, K., Yadmellat, P. and Chamorro, S. (2021) ‘Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning’, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3506–3511. doi:10.1109/iros51168.2021.9636480.
IEEE
K. Rezaee, P. Yadmellat, and S. Chamorro, “Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3506–3511, 2021, doi: 10.1109/iros51168.2021.9636480.
BibTeX
@inproceedings{rezaee2021motion,
title = {{Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning}},
author = {Rezaee, Kasra and Yadmellat, Peyman and Chamorro, Simon},
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year = {2021},
pages = {3506--3511},
doi = {10.1109/iros51168.2021.9636480},
url = {https://doi.org/10.1109/iros51168.2021.9636480}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2022 | 2 | |
| 2023 | 8 | |
| 2024 | 12 | |
| 2025 | 7 | |
| 2026 | 3 |
Its field-weighted citation impact is 1.59: it has been cited 1.59 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- motion planning
- autonomous vehicles
- uncertainty
- distributional reinforcement learning
- worst-case optimization
- occlusion
- sensing range
- limited field of view
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Rezaee, K., Yadmellat, P., & Chamorro, S. (2021). Motion Planning for Autonomous Vehicles in the Presence of Uncertainty Using Reinforcement Learning. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3506–3511. https://doi.org/10.1109/iros51168.2021.9636480
How many times has this paper been cited?
32 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Kasra Rezaee, Peyman Yadmellat, Simon Chamorro.
What is the DOI of this paper?
10.1109/iros51168.2021.9636480, permanently at https://doi.org/10.1109/iros51168.2021.9636480. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.