Journal article · 2026 · Open access
Q-Eval+: From Absolute to Relative Quality Evaluation for Text-to-Vision Content
IEEE Transactions on Pattern Analysis and Machine Intelligence, PP, 1–14 · Published 9 November 2026 · DOI 10.1109/tpami.2026.3751387
In brief
“Q-Eval+: From Absolute to Relative Quality Evaluation for Text-to-Vision Content” is a 2026 journal article by Zicheng Zhang and 7 co-authors, published in IEEE Transactions on Pattern Analysis and Machine Intelligence. OpenAlex counts 0 citations as of 18 November 2026. It is open access under a CC BY licence: the full text is free to read at IEEE Transactions on Pattern Analysis and Machine Intelligence.
- Citations
- 0
- Authors
- 8
Abstract
Text-to-vision (T2V) models are rapidly reshaping content creation, yet their outputs still require careful human screening before deployment in real applications. This creates an urgent need for evaluation tools that can 1) assign reliable absolute quality scores and 2) support fine-grained relative comparisons for model selection, reranking, and online A/B testing. Existing T2V benchmarks, however, are often limited by entangled definitions of visual quality and text-vision alignment, narrow coverage of images or videos, and the absence of systematically constructed pairwise and diagnostic annotations. Therefore, we present Q-Eval+, a large-scale dataset to jointly support absolute scoring and relative comparing for T2V content. Q-Eval+ consists of 220K image/video instances with 1.32M human annotations along two key dimensions: Q-Eval-Abs provides MOS-style labels for visual quality and alignment at both instance and model levels, while Q-Eval-Rel offers side-by-side preferences and choice-based VQA questions that reveal not only which instance is better, but also why. On top of Q-Eval+, we develop Q-Evaluator, a unified large multimodal model that bridges absolute and relative assessment through supervised fine-tuning with structured context prompts and a Vague-to-Specific alignment handling strategy. We further introduce symmetric and consistency accuracies to evaluate robustness under pairwise permutations. Extensive experiments show that Q-Evaluator achieves consistently strong correlations with human opinions and competitive comparative accuracy against existing open-source and proprietary LMM-based evaluators, providing a practical foundation for robust T2V quality control.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Zhang, Z., Wang, S., Wen, F., Li, C., Wang, W., Li, X., Zhai, G., & Liu, X. (2026). Q-Eval+: From Absolute to Relative Quality Evaluation for Text-to-Vision Content. IEEE Transactions on Pattern Analysis and Machine Intelligence, PP, 1–14. https://doi.org/10.1109/tpami.2026.3751387
MLA 9
Zhang, Zicheng, et al. “Q-Eval+: From Absolute to Relative Quality Evaluation for Text-to-Vision Content.” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PP, 2026, pp. 1–14. https://doi.org/10.1109/tpami.2026.3751387.
Chicago
Zhang, Zicheng, Shushi Wang, Farong Wen, Chunyi Li, Wei Wang, Xiaoyu Li, Guangtao Zhai, and Xiaohong Liu. 2026. “Q-Eval+: From Absolute to Relative Quality Evaluation for Text-to-Vision Content.” IEEE Transactions on Pattern Analysis and Machine Intelligence PP: 1–14. https://doi.org/10.1109/tpami.2026.3751387.
Harvard
Zhang, Z. et al. (2026) ‘Q-Eval+: From Absolute to Relative Quality Evaluation for Text-to-Vision Content’, IEEE Transactions on Pattern Analysis and Machine Intelligence, PP, pp. 1–14. doi:10.1109/tpami.2026.3751387.
IEEE
Z. Zhang et al., “Q-Eval+: From Absolute to Relative Quality Evaluation for Text-to-Vision Content,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PP, pp. 1–14, 2026, doi: 10.1109/tpami.2026.3751387.
BibTeX
@article{zhang2026eval,
title = {{Q-Eval+: From Absolute to Relative Quality Evaluation for Text-to-Vision Content}},
author = {Zhang, Zicheng and Wang, Shushi and Wen, Farong and Li, Chunyi and Wang, Wei and Li, Xiaoyu and Zhai, Guangtao and Liu, Xiaohong},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {PP},
pages = {1--14},
doi = {10.1109/tpami.2026.3751387},
url = {https://doi.org/10.1109/tpami.2026.3751387}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- visual quality assessment
- multimodal large language models
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Zhang, Z., Wang, S., Wen, F., Li, C., Wang, W., Li, X., Zhai, G., & Liu, X. (2026). Q-Eval+: From Absolute to Relative Quality Evaluation for Text-to-Vision Content. IEEE Transactions on Pattern Analysis and Machine Intelligence, PP, 1–14. https://doi.org/10.1109/tpami.2026.3751387
How many times has this paper been cited?
0 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at IEEE Transactions on Pattern Analysis and Machine Intelligence: sign in and open the full text from this page.
Who wrote this paper?
Zicheng Zhang, Shushi Wang, Farong Wen, Chunyi Li, Wei Wang, Xiaoyu Li, Guangtao Zhai, Xiaohong Liu.
What is the DOI of this paper?
10.1109/tpami.2026.3751387, permanently at https://doi.org/10.1109/tpami.2026.3751387. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.