KAPPA_COHEN

Updated 2023-11-06 14:44:13.247000

Syntax

SELECT [westclintech].[wct].[KAPPA_COHEN](
  <@InputData_RangeQuery, nvarchar(max),>
 ,<@RV, nvarchar(4000),>
 ,<@Weighting, nvarchar(4000),>)

Description

Use the scalar function KAPPA_COHEN to calculate Cohen’s kappa (κ) or weighted kappa as an index of inter-rater agreement between 2 raters. For weighted kappa, ratings must be ordinal. KAPPA_COHEN supports quadratic and linear weighting.

The equation for κ is

\kappa = \frac{P_a - P_c}{1-P_c}

Where:Pa is the relative observed agreement among ratersPc is the hypothetical probability of chance agreement

Arguments

@RV

the value to be returned by the function. Use the following values:

{"columns":[{"field":"'PA'"},{"field":"the relative observed agreement among raters"}],"rows":[{"'PA'":"'PC'","the relative observed agreement among raters":"the hypothetical probability of chance agreement"},{"'PA'":"'K'","the relative observed agreement among raters":"the test statistic"},{"'PA'":"'P'","the relative observed agreement among raters":"the p-value"},{"'PA'":"'Z'","the relative observed agreement among raters":"K / SE"},{"'PA'":"'SE'","the relative observed agreement among raters":"the standard error"},{"'PA'":"'NS'","the relative observed agreement among raters":"the number of subjects"},{"'PA'":"'NR'","the relative observed agreement among raters":"the number of ratings"}]}

@InputData_RangeQuery

a T-SQL statement, as a string, that specifies the rating 1- and rating 2- values.

@Weighting

identifies the weighting algorithm to be used. Permitted values are un-weighted, linear, and quadratic.

Return Type

float

Remarks

The function is insensitive to order; it does not matter what order the ratings are passed in.

Ratings are passed in as pairs.

For more than 2 raters, consider using KAPPA_FLEISS.

If @Weight is NULL, then @Weight = 'U'.

For quadratic weighting, set @Weight = 'Q'.

For linear weighting, set @Weight = 'L'.

Examples

SELECT *

INTO #k

FROM

(

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 1,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 2,

           1

    UNION ALL

    SELECT 3,

           1

    UNION ALL

    SELECT 3,

           1

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 1,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 2,

           2

    UNION ALL

    SELECT 3,

           2

    UNION ALL

    SELECT 3,

           2

    UNION ALL

    SELECT 3,

           2

    UNION ALL

    SELECT 3,

           2

    UNION ALL

    SELECT 3,

           2

    UNION ALL

    SELECT 3,

           2

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 1,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 2,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

    UNION ALL

    SELECT 3,

           3

) n(r1, r2);

SELECT p.stat,

       wct.KAPPA_COHEN('SELECT r1,r2 FROM #k', p.stat, 'U') k

FROM

(

    SELECT 'Pa'

    UNION ALL

    SELECT 'Pc'

    UNION ALL

    SELECT 'K'

    UNION ALL

    SELECT 'P'

    UNION ALL

    SELECT 'Z'

    UNION ALL

    SELECT 'SE'

    UNION ALL

    SELECT 'NS'

    UNION ALL

    SELECT 'NR'

) p(stat);

This produces the following result.

{"columns":[{"field":"stat"},{"field":"k","headerClass":"ag-right-aligned-header","cellClass":"ag-right-aligned-cell"}],"rows":[{"stat":"Pa","k":"0.7"},{"stat":"Pc","k":"0.41"},{"stat":"K","k":"0.491525423728813"},{"stat":"P","k":"3.19208256584873E-21"},{"stat":"Z","k":"9.45624243552736"},{"stat":"SE","k":"0.0519789363565954"},{"stat":"NS","k":"200"},{"stat":"NR","k":"3"}]}

Here an example using quadratic weighting.

with mycte (subj, rater, rating)

as (SELECT 1,

           'R1',

           'c'

    UNION ALL

    SELECT 1,

           'R2',

           'd'

    UNION ALL

    SELECT 2,

           'R1',

           'c'

    UNION ALL

    SELECT 2,

           'R2',

           'a'

    UNION ALL

    SELECT 3,

           'R1',

           'c'

    UNION ALL

    SELECT 3,

           'R2',

           'a'

    UNION ALL

    SELECT 4,

           'R1',

           'c'

    UNION ALL

    SELECT 4,

           'R2',

           'c'

    UNION ALL

    SELECT 5,

           'R1',

           'c'

    UNION ALL

    SELECT 5,

           'R2',

           'b'

    UNION ALL

    SELECT 6,

           'R1',

           'b'

    UNION ALL

    SELECT 6,

           'R2',

           'b'

    UNION ALL

    SELECT 7,

           'R1',

           'c'

    UNION ALL

    SELECT 7,

           'R2',

           'a'

    UNION ALL

    SELECT 8,

           'R1',

           'd'

    UNION ALL

    SELECT 8,

           'R2',

           'c'

    UNION ALL

    SELECT 9,

           'R1',

           'b'

    UNION ALL

    SELECT 9,

           'R2',

           'a'

    UNION ALL

    SELECT 10,

           'R1',

           'b'

    UNION ALL

    SELECT 10,

           'R2',

           'b'

    UNION ALL

    SELECT 11,

           'R1',

           'a'

    UNION ALL

    SELECT 11,

           'R2',

           'd'

    UNION ALL

    SELECT 12,

           'R1',

           'c'

    UNION ALL

    SELECT 12,

           'R2',

           'd'

    UNION ALL

    SELECT 13,

           'R1',

           'a'

    UNION ALL

    SELECT 13,

           'R2',

           'd'

    UNION ALL

    SELECT 14,

           'R1',

           'b'

    UNION ALL

    SELECT 14,

           'R2',

           'c'

    UNION ALL

    SELECT 15,

           'R1',

           'a'

    UNION ALL

    SELECT 15,

           'R2',

           'a'

    UNION ALL

    SELECT 16,

           'R1',

           'c'

    UNION ALL

    SELECT 16,

           'R2',

           'a'

    UNION ALL

    SELECT 17,

           'R1',

           'a'

    UNION ALL

    SELECT 17,

           'R2',

           'c'

    UNION ALL

    SELECT 18,

           'R1',

           'b'

    UNION ALL

    SELECT 18,

           'R2',

           'd'

    UNION ALL

    SELECT 19,

           'R1',

           'b'

    UNION ALL

    SELECT 19,

           'R2',

           'b'

    UNION ALL

    SELECT 20,

           'R1',

           'c'

    UNION ALL

    SELECT 20,

           'R2',

           'c'

    UNION ALL

    SELECT 21,

           'R1',

           'd'

    UNION ALL

    SELECT 21,

           'R2',

           'd'

    UNION ALL

    SELECT 22,

           'R1',

           'b'

    UNION ALL

    SELECT 22,

           'R2',

           'd'

    UNION ALL

    SELECT 23,

           'R1',

           'a'

    UNION ALL

    SELECT 23,

           'R2',

           'a'

    UNION ALL

    SELECT 24,

           'R1',

           'd'

    UNION ALL

    SELECT 24,

           'R2',

           'b'

    UNION ALL

    SELECT 25,

           'R1',

           'b'

    UNION ALL

    SELECT 25,

           'R2',

           'b')

SELECT m1.rating as r1,

       m2.rating as r2

INTO #c

FROM mycte m1

    JOIN mycte m2

        ON m1.subj = m2.subj

WHERE m1.rater = 'R1'

      AND m2.rater = 'R2';

SELECT p.*,

       wct.KAPPA_COHEN('SELECT r1,r2 FROM #C', p.stat, 'Q') k

FROM

(

    SELECT 'Pa'

    UNION ALL

    SELECT 'Pc'

    UNION ALL

    SELECT 'K'

    UNION ALL

    SELECT 'P'

    UNION ALL

    SELECT 'Z'

    UNION ALL

    SELECT 'SE'

    UNION ALL

    SELECT 'NS'

    UNION ALL

    SELECT 'NR'

) p(stat);

This produces the following result.

{"columns":[{"field":"stat"},{"field":"k","headerClass":"ag-right-aligned-header","cellClass":"ag-right-aligned-cell"}],"rows":[{"stat":"Pa","k":"0.751111111111111"},{"stat":"Pc","k":"0.749333333333333"},{"stat":"K","k":"0.00709219858156069"},{"stat":"P","k":"0.970935305534129"},{"stat":"Z","k":"0.0364352523330701"},{"stat":"SE","k":"0.194652105513855"},{"stat":"NS","k":"25"},{"stat":"NR","k":"4"}]}