Journal article · 2002 · Open access

Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers

Huiman Xie Barnhart, Michael J. Haber, Jingli Song

Biometrics, 58(4), 1020–1027 · Published 1 December 2002 · DOI 10.1111/j.0006-341x.2002.01020.x

Read the free full textCite this paper

In brief

“Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers” is a 2002 journal article by Huiman Xie Barnhart and 2 co-authors, published in Biometrics. OpenAlex counts 276 citations as of 16 November 2026, which puts it in the top 2% of works published in 2002 by citations. It is open access under a CC BY licence: the full text is free to read at Biometrics.

Citations
276
Most cited in
2024 (33)
Field-weighted impact
1.79
References
13
Authors
3

Abstract

Accurate and precise measurement is an important component of any proper study design. As elaborated by Lin (1989, Biometrics 45, 255-268), the concordance correlation coefficient (CCC) is more appropriate than other indices for measuring agreement when the variable of interest is continuous. However, this agreement index is defined in the context of comparing two fixed observers. In order to use multiple observers in a study involving large numbers of subjects, there is a need to assess agreement among these multiple observers. In this article, we present an overall CCC (OCCC) in terms of the interobserver variability for assessing agreement among multiple fixed observers. The OCCC turns out to be equivalent to the generalized CCC (King and Chinchilli, 2001, Statistics in Medicine 20, 2131-2147; Lin, 1989; Lin, 2000, Biometrics 56, 324-325) when the squared distance function is used. We evaluated the OCCC through generalized estimating equations (Barnhart and Williamson, 2001, Biometrics 57, 931-940) and U-statistics (King and Chinchilli, 2001) for inference. This article offers the following important points. First, it addresses the precision and accuracy indices as components of the OCCC. Second, it clarifies that the OCCC is the weighted average of all pairwise CCCs. Third, it is intuitively defined in terms of interobserver variability. Fourth, the inference approaches of GEE and the U-statistics are compared via simulations for small samples. Fifth, we illustrate the use of the OCCC by two medical examples with the GEE, U-statistics, and bootstrap approaches.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Barnhart, H. X., Haber, M. J., & Song, J. (2002). Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers. Biometrics, 58(4), 1020–1027. https://doi.org/10.1111/j.0006-341x.2002.01020.x

MLA 9

Barnhart, Huiman Xie, et al. “Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers.” Biometrics, vol. 58, no. 4, 2002, pp. 1020–1027. https://doi.org/10.1111/j.0006-341x.2002.01020.x.

Chicago

Barnhart, Huiman Xie, Michael J. Haber, and Jingli Song. 2002. “Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers.” Biometrics 58 (4): 1020–1027. https://doi.org/10.1111/j.0006-341x.2002.01020.x.

Harvard

Barnhart, H. X., Haber, M. J. and Song, J. (2002) ‘Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers’, Biometrics, 58(4), pp. 1020–1027. doi:10.1111/j.0006-341x.2002.01020.x.

IEEE

H. X. Barnhart, M. J. Haber, and J. Song, “Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers,” Biometrics, vol. 58, no. 4, pp. 1020–1027, 2002, doi: 10.1111/j.0006-341x.2002.01020.x.

BibTeX

@article{barnhart2002overall,
  title     = {{Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers}},
  author    = {Barnhart, Huiman Xie and Haber, Michael J. and Song, Jingli},
  journal   = {Biometrics},
  year      = {2002},
  volume    = {58},
  number    = {4},
  pages     = {1020--1027},
  doi       = {10.1111/j.0006-341x.2002.01020.x},
  url       = {https://doi.org/10.1111/j.0006-341x.2002.01020.x}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
20172121
20181515
20191313
20201717
202199
20221717
20231414
20243333
20251313
202699

Its field-weighted citation impact is 1.79: it has been cited 1.79 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement

  • concordance correlation coefficient
  • interobserver agreement
  • multiple observers
  • interobserver variability
  • precision and accuracy
  • generalized estimating equations
  • U-statistics
  • bootstrapping

Work with this paper

Most-cited papers in Decision Sciences

All highly cited Decision Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Barnhart, H. X., Haber, M. J., & Song, J. (2002). Overall Concordance Correlation Coefficient for Evaluating Agreement Among Multiple Observers. Biometrics, 58(4), 1020–1027. https://doi.org/10.1111/j.0006-341x.2002.01020.x

How many times has this paper been cited?

276 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at Biometrics.

Who wrote this paper?

Huiman Xie Barnhart, Michael J. Haber, Jingli Song.

What is the DOI of this paper?

10.1111/j.0006-341x.2002.01020.x. The permanent link is https://doi.org/10.1111/j.0006-341x.2002.01020.x.

Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.