Status of Weighted Agreement Statistics Between Two Raters in Ordinal Data Affected by Sample size and Number of Categories

Erdoğan, Semra | Sucu, Damla Hazal

The aim of this study is to introduce weighted inter-rater agreement statistics used in ordinal scales, compare weighted agreement statistics along with the Spearman correlation coefficient and revealtheir status of being affected by the sample size and number of categories. For this purpose, data for different sample sizes and number of categories were produced for the cases when there is no relationship or when there is low, medium, or high relationship between the two raters, and the aforementioned weighted agreement statistics were calculated. It can be said that Cohen’s kappa, Scott’s π, and Krippendorff’s alpha coefficients provide similar results with the correlation coefficients, and they get values very close to the correlation coefficient value in the B-P agreement statisti...