Preprint · 2025 · Open access

LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models

Ruilin Yao, Zhang, Bo, Jirui Huang, Xinwei Long, Yifang Zhang, Zou, Tianyu, Yu‐Fei Wu, Shichao Su, Yifan Xu, Zeng, Wenxi, Zhaoyu Yang, Li, Guoyou and 9 more

arXiv (Cornell University) · Published 21 May 2025 · DOI 10.48550/arxiv.2505.15616

Open PDFView at the publisherCite this paper

In brief

“LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models” is a 2025 preprint by Ruilin Yao and 20 co-authors, published in arXiv (Cornell University). OpenAlex counts 15 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
15
Most cited in
2026 (8)
References
60
Authors
21

Abstract

Multimodal Large Language Models (MLLMs) have achieved significant advances in integrating visual and linguistic information, yet their ability to reason about complex and real-world scenarios remains limited. The existing benchmarks are usually constructed in the task-oriented manner without guarantee that different task samples come from the same data distribution, thus they often fall short in evaluating the synergistic effects of lower-level perceptual capabilities on higher-order reasoning. To lift this limitation, we contribute Lens, a multi-level benchmark with 3.4K contemporary images and 60K+ human-authored questions covering eight tasks and 12 daily scenarios, forming three progressive task tiers, i.e., perception, understanding, and reasoning. One feature is that each image is equipped with rich annotations for all tasks. Thus, this dataset intrinsically supports to evaluate MLLMs to handle image-invariable prompts, from basic perception to compositional reasoning. In addition, our images are manully collected from the social media, in which 53% were published later than Jan. 2025. We evaluate 15+ frontier MLLMs such as Qwen2.5-VL-72B, InternVL3-78B, GPT-4o and two reasoning models QVQ-72B-preview and Kimi-VL. These models are released later than Dec. 2024, and none of them achieve an accuracy greater than 60% in the reasoning tasks. Project page: https://github.com/Lens4MLLMs/lens. ICCV 2025 workshop page: https://lens4mllms.github.io/mars2-workshop-iccv2025/

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Yao, R., Bo, Z., Huang, J., Long, X., Zhang, Y., Tianyu, Z., Wu, Y., Su, S., Xu, Y., Wenxi, Z., Yang, Z., Guoyou, L., Zhang, S., Li, Z., Chen, Y., Xiong, S., Xu, P., Zhang, J., Zhou, B., . . . Van Gool, L. (2025). LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2505.15616

MLA 9

Yao, Ruilin, et al. “LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2505.15616.

Chicago

Yao, Ruilin, Zhang, Bo, Jirui Huang, Xinwei Long, Yifang Zhang, Zou, Tianyu, Yu‐Fei Wu, et al. 2025. “LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2505.15616.

Harvard

Yao, R. et al. (2025) ‘LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models’, arXiv (Cornell University). doi:10.48550/arxiv.2505.15616.

IEEE

R. Yao et al., “LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2505.15616.

BibTeX

@misc{yao2025lens,
  title     = {{LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models}},
  author    = {Yao, Ruilin and Bo, Zhang, and Huang, Jirui and Long, Xinwei and Zhang, Yifang and Tianyu, Zou, and Wu, Yu‐Fei and Su, Shichao and Xu, Yifan and Wenxi, Zeng, and Yang, Zhaoyu and Guoyou, Li, and Zhang, Shilan and Li, Zichan and Chen, Yaxiong and Xiong, Shengwu and Xu, Peng and Zhang, Jiajun and Zhou, Bowen and Clifton, David Andrew and Van Gool, Luc},
  howpublished = {arXiv (Cornell University)},
  year      = {2025},
  doi       = {10.48550/arxiv.2505.15616},
  url       = {https://doi.org/10.48550/arxiv.2505.15616}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications

  • multimodal large language models
  • multimodal reasoning
  • benchmark dataset
  • compositional reasoning
  • image perception
  • multi-level evaluation
  • image annotation
  • social media images

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Yao, R., Bo, Z., Huang, J., Long, X., Zhang, Y., Tianyu, Z., Wu, Y., Su, S., Xu, Y., Wenxi, Z., Yang, Z., Guoyou, L., Zhang, S., Li, Z., Chen, Y., Xiong, S., Xu, P., Zhang, J., Zhou, B., . . . Van Gool, L. (2025). LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2505.15616

How many times has this paper been cited?

15 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Ruilin Yao, Zhang, Bo, Jirui Huang, Xinwei Long, Yifang Zhang, Zou, Tianyu, Yu‐Fei Wu, Shichao Su, Yifan Xu, Zeng, Wenxi, Zhaoyu Yang, Li, Guoyou and 9 more.

What is the DOI of this paper?

10.48550/arxiv.2505.15616, permanently at https://doi.org/10.48550/arxiv.2505.15616. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.