Journal article · 2026 · Open access

Complementary-Label Learning: A Survey of Methods, an Open-Source Toolkit, and Fair Benchmarking Results

Tan-Ha Mai, Nai-Xuan Ye, Wei Wang, Gang Niu, Takashi Ishida, Masashi Sugiyama, Hsuan-Tien Lin

ACM Computing Surveys · Published 3 October 2026 · DOI 10.1145/3847508

Open PDFView at the publisherCite this paper

In brief

“Complementary-Label Learning: A Survey of Methods, an Open-Source Toolkit, and Fair Benchmarking Results” is a 2026 journal article by Tan-Ha Mai and 6 co-authors, published in ACM Computing Surveys. OpenAlex counts 0 citations as of 16 November 2026. It is open access under a CC BY-NC-ND licence: the full text is free to read at ACM Computing Surveys.

Citations
0
Field-weighted impact
1.08
References
34
Authors
7

Abstract

Complementary-label learning (CLL) is a weakly supervised learning paradigm designed for multiclass classification tasks. In CLL, algorithms are provided with complementary labels, which indicate the classes an instance does not belong to, rather than specifying its true class. With the growing research interest in CLL and the unique challenges it presents, particularly in handling incomplete and indirect supervision, a comprehensive and systematic overview of existing methodologies, results, and research directions has become increasingly essential. In this survey, we provide the first systematic synthesis of recent advances in CLL and offer three major contributions. First, we establish a structured taxonomy that clarifies foundational concepts, distinctive algorithmic strategies, key contributions, and their inherent limitations. Second, to support reproducibility and accelerate future research, we introduce a publicly available open-source CLL toolkit that unifies key algorithm implementations and offers ready-to-use benchmarking pipelines. Third, we conduct an extensive benchmarking study covering a wide spectrum of CLL algorithms, using standardized evaluation protocols across both synthetic, human-annotated, and VLM-annotated complementary-label datasets. This provides the most complete performance comparison currently available. Finally, we discuss current open research challenges and identify promising avenues for future investigation, emphasizing improvements to the practical applicability and robustness of CLL in real-world scenarios. Overall, this survey provides an up-to-date, in-depth resource on the evolving CLL landscape.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Mai, T.-H., Ye, N.-X., Wang, W., Niu, G., Ishida, T., Sugiyama, M., & Lin, H.-T. (2026). Complementary-Label Learning: A Survey of Methods, an Open-Source Toolkit, and Fair Benchmarking Results. ACM Computing Surveys. https://doi.org/10.1145/3847508

MLA 9

Mai, Tan-Ha, et al. “Complementary-Label Learning: A Survey of Methods, an Open-Source Toolkit, and Fair Benchmarking Results.” ACM Computing Surveys, 2026. https://doi.org/10.1145/3847508.

Chicago

Mai, Tan-Ha, Nai-Xuan Ye, Wei Wang, Gang Niu, Takashi Ishida, Masashi Sugiyama, and Hsuan-Tien Lin. 2026. “Complementary-Label Learning: A Survey of Methods, an Open-Source Toolkit, and Fair Benchmarking Results.” ACM Computing Surveys. https://doi.org/10.1145/3847508.

Harvard

Mai, T.-H. et al. (2026) ‘Complementary-Label Learning: A Survey of Methods, an Open-Source Toolkit, and Fair Benchmarking Results’, ACM Computing Surveys. doi:10.1145/3847508.

IEEE

T.-H. Mai et al., “Complementary-Label Learning: A Survey of Methods, an Open-Source Toolkit, and Fair Benchmarking Results,” ACM Computing Surveys, 2026, doi: 10.1145/3847508.

BibTeX

@article{mai2026complementary,
  title     = {{Complementary-Label Learning: A Survey of Methods, an Open-Source Toolkit, and Fair Benchmarking Results}},
  author    = {Mai, Tan-Ha and Ye, Nai-Xuan and Wang, Wei and Niu, Gang and Ishida, Takashi and Sugiyama, Masashi and Lin, Hsuan-Tien},
  journal   = {ACM Computing Surveys},
  year      = {2026},
  doi       = {10.1145/3847508},
  url       = {https://doi.org/10.1145/3847508}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification

  • weakly supervised learning
  • multi-class classification
  • indirect supervision
  • open-source software toolkit
  • benchmarking
  • vision-language models
  • synthetic dataset
  • human-annotated dataset

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Mai, T.-H., Ye, N.-X., Wang, W., Niu, G., Ishida, T., Sugiyama, M., & Lin, H.-T. (2026). Complementary-Label Learning: A Survey of Methods, an Open-Source Toolkit, and Fair Benchmarking Results. ACM Computing Surveys. https://doi.org/10.1145/3847508

How many times has this paper been cited?

0 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at ACM Computing Surveys: sign in and open the PDF from this page.

Who wrote this paper?

Tan-Ha Mai, Nai-Xuan Ye, Wei Wang, Gang Niu, Takashi Ishida, Masashi Sugiyama, Hsuan-Tien Lin.

What is the DOI of this paper?

10.1145/3847508, permanently at https://doi.org/10.1145/3847508. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.