The calculation of the weighted kappa involves several steps:
Construct a contingency table that shows the frequency of each combination of ratings. Assign weights to each cell of the table. Weights are typically determined based on the distance between categories. Common weighting schemes include linear and quadratic weights. Calculate the observed agreement (O) and the expected agreement (E) based on the marginal totals of the table. Compute the weighted kappa using the formula: κ = (O - E) / (1 - E)
