Journal article · 2025 · Open access

DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving

Erfei Cui, Wenhai Wang, Zhiqi Li, Jiangwei Xie, Haoming Zou, Hanming Deng, Gen Luo, Lewei Lu, Xizhou Zhu, Jifeng Dai

Visual Intelligence, 3(1), 22 · Published 26 November 2025 · DOI 10.1007/s44267-025-00095-w

Open PDFView at the publisherCite this paper

In brief

“DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving” is a 2025 journal article by Erfei Cui and 9 co-authors, published in Visual Intelligence. OpenAlex counts 303 citations as of 21 November 2026. It is open access under a CC BY licence: the full text is free to read at Visual Intelligence.

Citations
303
Most cited in
2025 (120)
Field-weighted impact
42.62
References
103
Authors
10

Abstract

Large language models (LLMs) have opened up new possibilities for intelligent agents, endowing them with human-like thinking and cognitive abilities. In this work, we delve into the potential of large language models (LLMs) in autonomous driving (AD). We introduce DriveMLM, an LLM-based AD framework that can perform close-loop autonomous driving in realistic simulators. To this end, (1) we bridge the gap between the language decisions and the vehicle control commands by standardizing the decision states according to the off-the-shelf motion planning module. (2) We employ a multimodal LLM (MLLM) to model the behavior planning module of a module AD system, which uses driving rules, user commands, and inputs from various sensors (e.g., camera, LiDAR) as input and makes driving decisions and provide explanations. This model can plug-and-play in existing AD systems such as Autopilot and Apollo for close-loop driving. (3) We design an effective data engine to collect a dataset that includes decision state and corresponding explanation annotation for model training and evaluation. We conduct extensive experiments and show that replacing the decision-making modules of the Autopilot and Apollo with DriveMLM resulted in significant improvements of 3.2 and 4.7 points on the CARLA Town05 Long, respectively, demonstrating the effectiveness of our model. We hope this work can serve as a baseline for autonomous driving with LLMs.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Cui, E., Wang, W., Li, Z., Xie, J., Zou, H., Deng, H., Luo, G., Lu, L., Zhu, X., & Dai, J. (2025). DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving. Visual Intelligence, 3(1), 22. https://doi.org/10.1007/s44267-025-00095-w

MLA 9

Cui, Erfei, et al. “DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving.” Visual Intelligence, vol. 3, no. 1, 2025, p. 22. https://doi.org/10.1007/s44267-025-00095-w.

Chicago

Cui, Erfei, Wenhai Wang, Zhiqi Li, Jiangwei Xie, Haoming Zou, Hanming Deng, Gen Luo, Lewei Lu, Xizhou Zhu, and Jifeng Dai. 2025. “DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving.” Visual Intelligence 3 (1): 22. https://doi.org/10.1007/s44267-025-00095-w.

Harvard

Cui, E. et al. (2025) ‘DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving’, Visual Intelligence, 3(1), p. 22. doi:10.1007/s44267-025-00095-w.

IEEE

E. Cui et al., “DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving,” Visual Intelligence, vol. 3, no. 1, p. 22, 2025, doi: 10.1007/s44267-025-00095-w.

BibTeX

@article{cui2025drivemlm,
  title     = {{DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving}},
  author    = {Cui, Erfei and Wang, Wenhai and Li, Zhiqi and Xie, Jiangwei and Zou, Haoming and Deng, Hanming and Luo, Gen and Lu, Lewei and Zhu, Xizhou and Dai, Jifeng},
  journal   = {Visual Intelligence},
  year      = {2025},
  volume    = {3},
  number    = {1},
  pages     = {22},
  doi       = {10.1007/s44267-025-00095-w},
  url       = {https://doi.org/10.1007/s44267-025-00095-w}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202311
20246565
2025120120
2026117117

Its field-weighted citation impact is 42.62: it has been cited 42.62 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety

  • large language models
  • autonomous driving
  • behavior planning
  • motion planning
  • CARLA simulator
  • autopilot
  • Apollo

Work with this paper

Most-cited papers in Engineering

All highly cited Engineering papers

Questions about this paper

How do I cite this paper in APA style?

Cui, E., Wang, W., Li, Z., Xie, J., Zou, H., Deng, H., Luo, G., Lu, L., Zhu, X., & Dai, J. (2025). DriveMLM: aligning multi-modal large language models with behavioral planning states for autonomous driving. Visual Intelligence, 3(1), 22. https://doi.org/10.1007/s44267-025-00095-w

How many times has this paper been cited?

303 times, according to OpenAlex on 21 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Visual Intelligence: sign in and open the PDF from this page.

Who wrote this paper?

Erfei Cui, Wenhai Wang, Zhiqi Li, Jiangwei Xie, Haoming Zou, Hanming Deng, Gen Luo, Lewei Lu, Xizhou Zhu, Jifeng Dai.

What is the DOI of this paper?

10.1007/s44267-025-00095-w, permanently at https://doi.org/10.1007/s44267-025-00095-w. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 21 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.