Journal article · 2023 · Open access

Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving

Weitao Zhou, Zhong Liang Cao, Nanshan Deng, Kun Jiang, Diange Yang

IEEE Transactions on Intelligent Transportation Systems, 24(8), 7932–7942 · Published 18 April 2023 · DOI 10.1109/tits.2023.3266885

Open PDFView at the publisherCite this paper

In brief

“Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving” is a 2023 journal article by Weitao Zhou and 4 co-authors, published in IEEE Transactions on Intelligent Transportation Systems. OpenAlex counts 30 citations as of 22 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
30
Most cited in
2025 (14)
Field-weighted impact
1.65
References
40
Authors
5

Abstract

Deep reinforcement learning (DRL) has emerged as a promising approach for developing more intelligent autonomous vehicles (AVs). A typical DRL application on AVs is to train a neural network-based driving policy. However, the black-box nature of neural networks can result in unpredictable decision failures, making such AVs unreliable. To this end, this work proposes a method to identify and protect unreliable decisions of a DRL driving policy. The basic idea is to estimate and constrain the policy’s performance uncertainty, which quantifies potential performance drop due to insufficient training data or network fitting errors. By constraining the uncertainty, the DRL model’s performance is always greater than that of a baseline policy. The uncertainty caused by insufficient data is estimated by the bootstrapped method. Then, the uncertainty caused by the network fitting error is estimated using an ensemble network. Finally, a baseline policy is added as the performance lower bound to avoid potential decision failures. The overall framework is called uncertainty-bound reinforcement learning (UBRL). The proposed UBRL is evaluated on DRL policies with different amounts of training data, taking an unprotected left-turn driving case as an example. The result shows that the UBRL method can identify potentially unreliable decisions of DRL policy. The UBRL guarantees to outperform baseline policy even when the DRL policy is not well-trained and has high uncertainty. Meanwhile, the performance of UBRL improves with more training data. Such a method is valuable for the DRL application on real-road driving and provides a metric to evaluate a DRL policy.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Zhou, W., Cao, Z. L., Deng, N., Jiang, K., & Yang, D. (2023). Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving. IEEE Transactions on Intelligent Transportation Systems, 24(8), 7932–7942. https://doi.org/10.1109/tits.2023.3266885

MLA 9

Zhou, Weitao, et al. “Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving.” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 8, 2023, pp. 7932–7942. https://doi.org/10.1109/tits.2023.3266885.

Chicago

Zhou, Weitao, Zhong Liang Cao, Nanshan Deng, Kun Jiang, and Diange Yang. 2023. “Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving.” IEEE Transactions on Intelligent Transportation Systems 24 (8): 7932–7942. https://doi.org/10.1109/tits.2023.3266885.

Harvard

Zhou, W. et al. (2023) ‘Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving’, IEEE Transactions on Intelligent Transportation Systems, 24(8), pp. 7932–7942. doi:10.1109/tits.2023.3266885.

IEEE

W. Zhou, Z. L. Cao, N. Deng, K. Jiang, and D. Yang, “Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 8, pp. 7932–7942, 2023, doi: 10.1109/tits.2023.3266885.

BibTeX

@article{zhou2023identify,
  title     = {{Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving}},
  author    = {Zhou, Weitao and Cao, Zhong Liang and Deng, Nanshan and Jiang, Kun and Yang, Diange},
  journal   = {IEEE Transactions on Intelligent Transportation Systems},
  year      = {2023},
  volume    = {24},
  number    = {8},
  pages     = {7932--7942},
  doi       = {10.1109/tits.2023.3266885},
  url       = {https://doi.org/10.1109/tits.2023.3266885}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202311
202488
20251414
202677

Its field-weighted citation impact is 1.65: it has been cited 1.65 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics

  • deep reinforcement learning
  • autonomous driving
  • uncertainty estimation
  • bootstrapping
  • neural network ensembles
  • limited training data

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Zhou, W., Cao, Z. L., Deng, N., Jiang, K., & Yang, D. (2023). Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving. IEEE Transactions on Intelligent Transportation Systems, 24(8), 7932–7942. https://doi.org/10.1109/tits.2023.3266885

How many times has this paper been cited?

30 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Weitao Zhou, Zhong Liang Cao, Nanshan Deng, Kun Jiang, Diange Yang.

What is the DOI of this paper?

10.1109/tits.2023.3266885, permanently at https://doi.org/10.1109/tits.2023.3266885. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.