While kappa is a useful statistic, it has some limitations. One significant issue is its sensitivity to the prevalence of the observed categories. When one category is much more common than others, kappa can give a misleading indication of agreement. Additionally, kappa does not account for the possibility that some disagreements might be more critical than others, which is where weighted kappa might be more appropriate.
