Skip to contents

Cohen's kappa for two raters

reliability_kappa.Rd

Native implementation of Cohen's kappa for nominal-scale agreement between two raters (Cohen, 1960). Unweighted (Eq. 1) and weighted (linear or quadratic) variants are supported.

Usage

reliability_kappa(observations, weight = c("unweighted", "equal", "squared"))

Arguments

observations

A subjects x 2 raters matrix or data.frame. Rows are units; the two columns are the two raters. Must not contain NA.

weight

Weighting scheme for disagreements:

"unweighted"

All disagreements equally serious (default).

"equal"

Linear weights: 1 - |i - j|/(k - 1).

"squared"

Quadratic weights: 1 - ((i - j)/(k - 1))^2.

Value

A list with elements method, value, ci_lower, ci_upper, per_value, n_observers, n_units, n_pairable. ci_lower and ci_upper are populated for unweighted kappa using the asymptotic standard error from Cohen (1960, Eq. 7); NA for weighted variants. per_value (unweighted only) gives per-category kappa via dichotomisation.

References

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46. doi:10.1177/001316446002000104