Preprint · 2025 · Open access
LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models
arXiv (Cornell University) · Published 21 May 2025 · DOI 10.48550/arxiv.2505.15616
In brief
“LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models” is a 2025 preprint by Ruilin Yao and 20 co-authors, published in arXiv (Cornell University). OpenAlex counts 15 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 15
- Most cited in
- 2026 (8)
- References
- 60
- Authors
- 21
Abstract
Multimodal Large Language Models (MLLMs) have achieved significant advances in integrating visual and linguistic information, yet their ability to reason about complex and real-world scenarios remains limited. The existing benchmarks are usually constructed in the task-oriented manner without guarantee that different task samples come from the same data distribution, thus they often fall short in evaluating the synergistic effects of lower-level perceptual capabilities on higher-order reasoning. To lift this limitation, we contribute Lens, a multi-level benchmark with 3.4K contemporary images and 60K+ human-authored questions covering eight tasks and 12 daily scenarios, forming three progressive task tiers, i.e., perception, understanding, and reasoning. One feature is that each image is equipped with rich annotations for all tasks. Thus, this dataset intrinsically supports to evaluate MLLMs to handle image-invariable prompts, from basic perception to compositional reasoning. In addition, our images are manully collected from the social media, in which 53% were published later than Jan. 2025. We evaluate 15+ frontier MLLMs such as Qwen2.5-VL-72B, InternVL3-78B, GPT-4o and two reasoning models QVQ-72B-preview and Kimi-VL. These models are released later than Dec. 2024, and none of them achieve an accuracy greater than 60% in the reasoning tasks. Project page: https://github.com/Lens4MLLMs/lens. ICCV 2025 workshop page: https://lens4mllms.github.io/mars2-workshop-iccv2025/
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Yao, R., Bo, Z., Huang, J., Long, X., Zhang, Y., Tianyu, Z., Wu, Y., Su, S., Xu, Y., Wenxi, Z., Yang, Z., Guoyou, L., Zhang, S., Li, Z., Chen, Y., Xiong, S., Xu, P., Zhang, J., Zhou, B., . . . Van Gool, L. (2025). LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2505.15616
MLA 9
Yao, Ruilin, et al. “LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2505.15616.
Chicago
Yao, Ruilin, Zhang, Bo, Jirui Huang, Xinwei Long, Yifang Zhang, Zou, Tianyu, Yu‐Fei Wu, et al. 2025. “LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2505.15616.
Harvard
Yao, R. et al. (2025) ‘LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models’, arXiv (Cornell University). doi:10.48550/arxiv.2505.15616.
IEEE
R. Yao et al., “LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2505.15616.
BibTeX
@misc{yao2025lens,
title = {{LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models}},
author = {Yao, Ruilin and Bo, Zhang, and Huang, Jirui and Long, Xinwei and Zhang, Yifang and Tianyu, Zou, and Wu, Yu‐Fei and Su, Shichao and Xu, Yifan and Wenxi, Zeng, and Yang, Zhaoyu and Guoyou, Li, and Zhang, Shilan and Li, Zichan and Chen, Yaxiong and Xiong, Shengwu and Xu, Peng and Zhang, Jiajun and Zhou, Bowen and Clifton, David Andrew and Van Gool, Luc},
howpublished = {arXiv (Cornell University)},
year = {2025},
doi = {10.48550/arxiv.2505.15616},
url = {https://doi.org/10.48550/arxiv.2505.15616}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- multimodal large language models
- multimodal reasoning
- benchmark dataset
- compositional reasoning
- image perception
- multi-level evaluation
- image annotation
- social media images
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Yao, R., Bo, Z., Huang, J., Long, X., Zhang, Y., Tianyu, Z., Wu, Y., Su, S., Xu, Y., Wenxi, Z., Yang, Z., Guoyou, L., Zhang, S., Li, Z., Chen, Y., Xiong, S., Xu, P., Zhang, J., Zhou, B., . . . Van Gool, L. (2025). LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2505.15616
How many times has this paper been cited?
15 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Ruilin Yao, Zhang, Bo, Jirui Huang, Xinwei Long, Yifang Zhang, Zou, Tianyu, Yu‐Fei Wu, Shichao Su, Yifan Xu, Zeng, Wenxi, Zhaoyu Yang, Li, Guoyou and 9 more.
What is the DOI of this paper?
10.48550/arxiv.2505.15616, permanently at https://doi.org/10.48550/arxiv.2505.15616. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.