Journal article · 2025 · Open access

Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections

Zengqi Peng, Yubin Wang, Lei Zheng, Jun Ma

IEEE Transactions on Vehicular Technology, 74(6), 8824–8838 · Published 12 February 2025 · DOI 10.1109/tvt.2025.3541401

Open PDFView at the publisherCite this paper

In brief

“Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections” is a 2025 journal article by Zengqi Peng and 3 co-authors, published in IEEE Transactions on Vehicular Technology. OpenAlex counts 14 citations as of 22 November 2026, which puts it in the top 2% of works published in 2025 by citations. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
14
Most cited in
2026 (8)
Field-weighted impact
2.45
References
42
Authors
4

Abstract

In this work, we present BiM-ACPPO, a bilevel multi-armed bandit-based hierarchical reinforcement learning framework for interaction-aware decision-making and planning at unsignalized intersections. Essentially, it proactively takes the uncertainties associated with surrounding vehicles (SVs) into consideration, which encompass those stemming from the driver's intention, interactive behaviors, and the varying number of SVs. Intermediate decision variables are introduced to enable the high-level RL policy to provide an interaction-aware reference, for guiding low-level model predictive control (MPC) and further enhancing the generalization ability of the proposed framework. By leveraging the structured nature of self-driving at unsignalized intersections, the training problem of the RL policy is modeled as a bilevel curriculum learning task, which is addressed by the proposed Exp3.S-based BiMAB algorithm. It is noteworthy that the training curricula are dynamically adjusted, thereby facilitating the sample efficiency of the RL training process. Comparative experiments are conducted in the high-fidelity CARLA simulator, and the results indicate that our approach achieves superior performance compared to all baseline methods. Furthermore, experimental results in two new urban driving scenarios clearly demonstrate the commendable generalization performance of the proposed method.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Peng, Z., Wang, Y., Zheng, L., & Ma, J. (2025). Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections. IEEE Transactions on Vehicular Technology, 74(6), 8824–8838. https://doi.org/10.1109/tvt.2025.3541401

MLA 9

Peng, Zengqi, et al. “Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections.” IEEE Transactions on Vehicular Technology, vol. 74, no. 6, 2025, pp. 8824–8838. https://doi.org/10.1109/tvt.2025.3541401.

Chicago

Peng, Zengqi, Yubin Wang, Lei Zheng, and Jun Ma. 2025. “Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections.” IEEE Transactions on Vehicular Technology 74 (6): 8824–8838. https://doi.org/10.1109/tvt.2025.3541401.

Harvard

Peng, Z. et al. (2025) ‘Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections’, IEEE Transactions on Vehicular Technology, 74(6), pp. 8824–8838. doi:10.1109/tvt.2025.3541401.

IEEE

Z. Peng, Y. Wang, L. Zheng, and J. Ma, “Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections,” IEEE Transactions on Vehicular Technology, vol. 74, no. 6, pp. 8824–8838, 2025, doi: 10.1109/tvt.2025.3541401.

BibTeX

@article{peng2025bilevel,
  title     = {{Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections}},
  author    = {Peng, Zengqi and Wang, Yubin and Zheng, Lei and Ma, Jun},
  journal   = {IEEE Transactions on Vehicular Technology},
  year      = {2025},
  volume    = {74},
  number    = {6},
  pages     = {8824--8838},
  doi       = {10.1109/tvt.2025.3541401},
  url       = {https://doi.org/10.1109/tvt.2025.3541401}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety

  • bi-level optimization
  • multi-armed bandits
  • hierarchical reinforcement learning
  • autonomous vehicles
  • unsignalized intersections
  • model predictive control
  • curriculum learning
  • driver intention

Work with this paper

Most-cited papers in Engineering

All highly cited Engineering papers

Questions about this paper

How do I cite this paper in APA style?

Peng, Z., Wang, Y., Zheng, L., & Ma, J. (2025). Bilevel Multi-Armed Bandit-Based Hierarchical Reinforcement Learning for Interaction-Aware Self-Driving at Unsignalized Intersections. IEEE Transactions on Vehicular Technology, 74(6), 8824–8838. https://doi.org/10.1109/tvt.2025.3541401

How many times has this paper been cited?

14 times, according to OpenAlex on 22 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Zengqi Peng, Yubin Wang, Lei Zheng, Jun Ma.

What is the DOI of this paper?

10.1109/tvt.2025.3541401, permanently at https://doi.org/10.1109/tvt.2025.3541401. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 22 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.