Preprint · 2025 · Open access

VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity

Bi, Jing, Guo, Junjia, Susan Liang, Sun, Guangyu, Luchuan Song, Tang, Yunlong, He, Jinxi, Jingwen Wu, Ali Vosoughi, Chen Chen, Xu, Chenliang

arXiv (Cornell University) · Published 14 March 2025 · DOI 10.48550/arxiv.2503.11557

Open PDFView at the publisherCite this paper

In brief

“VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity” is a 2025 preprint by Bi, Jing and 10 co-authors, published in arXiv (Cornell University). OpenAlex counts 28 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at arXiv (Cornell University).

Citations
28
Most cited in
2025 (16)
References
51
Authors
11

Abstract

Visual reasoning is central to human cognition, enabling individuals to interpret and abstractly understand their environment. Although recent Multimodal Large Language Models (MLLMs) have demonstrated impressive performance across language and vision-language tasks, existing benchmarks primarily measure recognition-based skills and inadequately assess true visual reasoning capabilities. To bridge this critical gap, we introduce VERIFY, a benchmark explicitly designed to isolate and rigorously evaluate the visual reasoning capabilities of state-of-the-art MLLMs. VERIFY compels models to reason primarily from visual information, providing minimal textual context to reduce reliance on domain-specific knowledge and linguistic biases. Each problem is accompanied by a human-annotated reasoning path, making it the first to provide in-depth evaluation of model decision-making processes. Additionally, we propose novel metrics that assess visual reasoning fidelity beyond mere accuracy, highlighting critical imbalances in current model reasoning patterns. Our comprehensive benchmarking of leading MLLMs uncovers significant limitations, underscoring the need for a balanced and holistic approach to both perception and reasoning. For more teaser and testing, visit our project page (https://verify-eqh.pages.dev/).

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Jing, B., Junjia, G., Liang, S., Guangyu, S., Song, L., Yunlong, T., Jinxi, H., Wu, J., Vosoughi, A., Chen, C., & Chenliang, X. (2025). VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2503.11557

MLA 9

Jing, Bi,, et al. “VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity.” arXiv (Cornell University), 2025. https://doi.org/10.48550/arxiv.2503.11557.

Chicago

Jing, Bi,, Guo, Junjia, Susan Liang, Sun, Guangyu, Luchuan Song, Tang, Yunlong, He, Jinxi, et al. 2025. “VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2503.11557.

Harvard

Jing, B. et al. (2025) ‘VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity’, arXiv (Cornell University). doi:10.48550/arxiv.2503.11557.

IEEE

B. Jing et al., “VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity,” arXiv (Cornell University), 2025, doi: 10.48550/arxiv.2503.11557.

BibTeX

@misc{jing2025verify,
  title     = {{VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity}},
  author    = {Jing, Bi, and Junjia, Guo, and Liang, Susan and Guangyu, Sun, and Song, Luchuan and Yunlong, Tang, and Jinxi, He, and Wu, Jingwen and Vosoughi, Ali and Chen, Chen and Chenliang, Xu,},
  howpublished = {arXiv (Cornell University)},
  year      = {2025},
  doi       = {10.48550/arxiv.2503.11557},
  url       = {https://doi.org/10.48550/arxiv.2503.11557}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications

  • visual reasoning
  • multimodal large language models
  • benchmark
  • visual explanations
  • evaluation metrics
  • language bias

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Jing, B., Junjia, G., Liang, S., Guangyu, S., Song, L., Yunlong, T., Jinxi, H., Wu, J., Vosoughi, A., Chen, C., & Chenliang, X. (2025). VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2503.11557

How many times has this paper been cited?

28 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Bi, Jing, Guo, Junjia, Susan Liang, Sun, Guangyu, Luchuan Song, Tang, Yunlong, He, Jinxi, Jingwen Wu, Ali Vosoughi, Chen Chen, Xu, Chenliang.

What is the DOI of this paper?

10.48550/arxiv.2503.11557, permanently at https://doi.org/10.48550/arxiv.2503.11557. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.