Journal article · 2002 · Open access
Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers
Biometrics, 58(4), 1020–1027 · Published 1 December 2002 · DOI 10.1111/j.0006-341x.2002.01020.x
In brief
“Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers” is a 2002 journal article by Huiman Xie Barnhart and 2 co-authors, published in Biometrics. OpenAlex counts 276 citations as of 16 November 2026, which puts it in the top 2% of works published in 2002 by citations. It is open access under a CC BY licence: the full text is free to read at Biometrics.
- Citations
- 276
- Most cited in
- 2024 (33)
- Field-weighted impact
- 1.79
- References
- 13
- Authors
- 3
Abstract
Accurate and precise measurement is an important component of any proper study design. As elaborated by Lin (1989, Biometrics 45, 255-268), the concordance correlation coefficient (CCC) is more appropriate than other indices for measuring agreement when the variable of interest is continuous. However, this agreement index is defined in the context of comparing two fixed observers. In order to use multiple observers in a study involving large numbers of subjects, there is a need to assess agreement among these multiple observers. In this article, we present an overall CCC (OCCC) in terms of the interobserver variability for assessing agreement among multiple fixed observers. The OCCC turns out to be equivalent to the generalized CCC (King and Chinchilli, 2001, Statistics in Medicine 20, 2131-2147; Lin, 1989; Lin, 2000, Biometrics 56, 324-325) when the squared distance function is used. We evaluated the OCCC through generalized estimating equations (Barnhart and Williamson, 2001, Biometrics 57, 931-940) and U-statistics (King and Chinchilli, 2001) for inference. This article offers the following important points. First, it addresses the precision and accuracy indices as components of the OCCC. Second, it clarifies that the OCCC is the weighted average of all pairwise CCCs. Third, it is intuitively defined in terms of interobserver variability. Fourth, the inference approaches of GEE and the U-statistics are compared via simulations for small samples. Fifth, we illustrate the use of the OCCC by two medical examples with the GEE, U-statistics, and bootstrap approaches.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Barnhart, H. X., Haber, M. J., & Song, J. (2002). Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers. Biometrics, 58(4), 1020–1027. https://doi.org/10.1111/j.0006-341x.2002.01020.x
MLA 9
Barnhart, Huiman Xie, et al. “Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers.” Biometrics, vol. 58, no. 4, 2002, pp. 1020–1027. https://doi.org/10.1111/j.0006-341x.2002.01020.x.
Chicago
Barnhart, Huiman Xie, Michael J. Haber, and Jingli Song. 2002. “Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers.” Biometrics 58 (4): 1020–1027. https://doi.org/10.1111/j.0006-341x.2002.01020.x.
Harvard
Barnhart, H. X., Haber, M. J. and Song, J. (2002) ‘Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers’, Biometrics, 58(4), pp. 1020–1027. doi:10.1111/j.0006-341x.2002.01020.x.
IEEE
H. X. Barnhart, M. J. Haber, and J. Song, “Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers,” Biometrics, vol. 58, no. 4, pp. 1020–1027, 2002, doi: 10.1111/j.0006-341x.2002.01020.x.
BibTeX
@article{barnhart2002overall,
title = {{Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers}},
author = {Barnhart, Huiman Xie and Haber, Michael J. and Song, Jingli},
journal = {Biometrics},
year = {2002},
volume = {58},
number = {4},
pages = {1020--1027},
doi = {10.1111/j.0006-341x.2002.01020.x},
url = {https://doi.org/10.1111/j.0006-341x.2002.01020.x}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2017 | 21 | |
| 2018 | 15 | |
| 2019 | 13 | |
| 2020 | 17 | |
| 2021 | 9 | |
| 2022 | 17 | |
| 2023 | 14 | |
| 2024 | 33 | |
| 2025 | 13 | |
| 2026 | 9 |
Its field-weighted citation impact is 1.79: it has been cited 1.79 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- concordance correlation coefficient
- interobserver agreement
- multiple observers
- interobserver variability
- precision and accuracy
- generalized estimating equations
- U-statistics
- bootstrapping
Work with this paper
Most-cited papers in Decision Sciences
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsMatthew James Page, Joanne E. McKenzie, Patrick M. Bossuyt and 23 more · BMJ · 2021 · 144,919 citations
- Evaluating Structural Equation Models with Unobservable Variables and Measurement ErrorClaes Fornell, David F. Larcker · Journal of Marketing Research · 1981 · 118,336 citations
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesLi‐tze Hu, Peter M. Bentler · Structural Equation Modeling A Multidisciplinary Journal · 1999 · 117,461 citations
- Common method biases in behavioral research: A critical review of the literature and recommended remedies.Philip M. Podsakoff, SCOTT B. MACKENZIE, Jeong Yeon Lee and 1 more · Journal of Applied Psychology · 2003 · 89,248 citations
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statementDavid Moher, Alessandro Liberati, Jennifer Tetzlaff and 2 more · BMJ · 2009 · 85,524 citations
Questions about this paper
How do I cite this paper in APA style?
Barnhart, H. X., Haber, M. J., & Song, J. (2002). Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers. Biometrics, 58(4), 1020–1027. https://doi.org/10.1111/j.0006-341x.2002.01020.x
How many times has this paper been cited?
276 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access: read the free full text at Biometrics.
Who wrote this paper?
Huiman Xie Barnhart, Michael J. Haber, Jingli Song.
What is the DOI of this paper?
10.1111/j.0006-341x.2002.01020.x. The permanent link is https://doi.org/10.1111/j.0006-341x.2002.01020.x.
Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.