Journal article · 2018 · Open access

Asymptotic variability of (multilevel) multirater kappa coefficients

Sophie Vanbelle

Statistical Methods in Medical Research, 28(10-11), 3012–3026 · Published 22 August 2018 · DOI 10.1177/0962280218794733

Open PDFView at the publisherCite this paper

In brief

“Asymptotic variability of (multilevel) multirater kappa coefficients” is a 2018 journal article by Sophie Vanbelle, published in Statistical Methods in Medical Research. OpenAlex counts 35 citations as of 18 November 2026. It is open access under a CC BY-NC licence: the full text is free to read at Statistical Methods in Medical Research.

Citations
35
Most cited in
2024 (8)
Field-weighted impact
0.72
References
28
Authors
1

Abstract

Agreement studies are of paramount importance in various scientific domains. When several observers classify objects on categorical scales, agreement can be quantified through multirater kappa coefficients. In most statistical packages, the standard error of these coefficients is only available under the null hypothesis that the coefficient is equal to zero, preventing the construction of confidence intervals in the general case. The aim of this paper is triple. First, simple analytic formulae for the standard error of multirater kappa coefficients will be given in the general case. Second, these formulae will be extended to the case of multilevel data structures. The formulae are based on simple matrix algebra and are implemented in the R package "multiagree". Third, guidelines on the choice between the different mulitrater kappa coefficients will be provided.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Vanbelle, S. (2018). Asymptotic variability of (multilevel) multirater kappa coefficients. Statistical Methods in Medical Research, 28(10-11), 3012–3026. https://doi.org/10.1177/0962280218794733

MLA 9

Vanbelle, Sophie. “Asymptotic variability of (multilevel) multirater kappa coefficients.” Statistical Methods in Medical Research, vol. 28, no. 10-11, 2018, pp. 3012–3026. https://doi.org/10.1177/0962280218794733.

Chicago

Vanbelle, Sophie. 2018. “Asymptotic variability of (multilevel) multirater kappa coefficients.” Statistical Methods in Medical Research 28 (10-11): 3012–3026. https://doi.org/10.1177/0962280218794733.

Harvard

Vanbelle, S. (2018) ‘Asymptotic variability of (multilevel) multirater kappa coefficients’, Statistical Methods in Medical Research, 28(10-11), pp. 3012–3026. doi:10.1177/0962280218794733.

IEEE

S. Vanbelle, “Asymptotic variability of (multilevel) multirater kappa coefficients,” Statistical Methods in Medical Research, vol. 28, no. 10-11, pp. 3012–3026, 2018, doi: 10.1177/0962280218794733.

BibTeX

@article{vanbelle2018asymptotic,
  title     = {{Asymptotic variability of (multilevel) multirater kappa coefficients}},
  author    = {Vanbelle, Sophie},
  journal   = {Statistical Methods in Medical Research},
  year      = {2018},
  volume    = {28},
  number    = {10-11},
  pages     = {3012--3026},
  doi       = {10.1177/0962280218794733},
  url       = {https://doi.org/10.1177/0962280218794733}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
201911
202044
202155
202233
202355
202488
202555
202644

Its field-weighted citation impact is 0.72: it has been cited 0.72 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement

  • interobserver agreement
  • standard errors
  • confidence intervals
  • multilevel data
  • categorical classification
  • closed-form expressions

Work with this paper

Most-cited papers in Decision Sciences

All highly cited Decision Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Vanbelle, S. (2018). Asymptotic variability of (multilevel) multirater kappa coefficients. Statistical Methods in Medical Research, 28(10-11), 3012–3026. https://doi.org/10.1177/0962280218794733

How many times has this paper been cited?

35 times, according to OpenAlex on 18 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Statistical Methods in Medical Research: sign in and open the PDF from this page.

Who wrote this paper?

Sophie Vanbelle.

What is the DOI of this paper?

10.1177/0962280218794733, permanently at https://doi.org/10.1177/0962280218794733. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 18 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.