-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathkrippendorff_alpha_results.txt
121 lines (107 loc) · 2.69 KB
/
krippendorff_alpha_results.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
R version 3.3.3 (2017-03-06) -- "Another Canoe"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> #---
> #title: "Krif test"
> #author: "YZ"
> #date: "April 4, 2017"
> #output: html_document
> #---
> #
> #```{r}
>
> # encoding: 0 for none, 1 for some tools described, 2 for available upon request
> # only 3 levels of encoding for testing (that we actually used)
>
> require(readr, quietly = TRUE)
> krip_alpha <- read_csv("./krip\ alpha.csv",col_names = FALSE)
Parsed with column specification:
cols(
X1 = col_integer(),
X2 = col_integer(),
X3 = col_integer()
)
>
> head(krip_alpha, 10)
# A tibble: 10 × 3
X1 X2 X3
<int> <int> <int>
1 1 1 1
2 1 1 1
3 0 0 0
4 0 0 0
5 1 1 1
6 0 0 0
7 0 0 0
8 1 1 1
9 1 1 1
10 1 1 1
>
> library(irr, quietly = TRUE)
>
> # 3 coders tests:
> kripp.alpha(data.matrix(t(krip_alpha)),"ordinal")
Krippendorff's alpha
Subjects = 30
Raters = 3
alpha = 0.775
>
> kripp.alpha(data.matrix(t(krip_alpha)),"nominal")
Krippendorff's alpha
Subjects = 30
Raters = 3
alpha = 0.77
>
> # MK&YZ
> kripp.alpha(data.matrix(t(krip_alpha))[1:2,],"ordinal")
Krippendorff's alpha
Subjects = 30
Raters = 2
alpha = 0.755
> kripp.alpha(data.matrix(t(krip_alpha))[1:2,],"nominal")
Krippendorff's alpha
Subjects = 30
Raters = 2
alpha = 0.69
>
>
> # MK&DW
> kripp.alpha(data.matrix(t(krip_alpha))[-2,],"ordinal")
Krippendorff's alpha
Subjects = 30
Raters = 2
alpha = 0.702
> kripp.alpha(data.matrix(t(krip_alpha))[-2,],"nominal")
Krippendorff's alpha
Subjects = 30
Raters = 2
alpha = 0.752
>
>
> # YZ&DW
> kripp.alpha(data.matrix(t(krip_alpha))[-1,],"ordinal")
Krippendorff's alpha
Subjects = 30
Raters = 2
alpha = 0.88
> kripp.alpha(data.matrix(t(krip_alpha))[-1,],"nominal")
Krippendorff's alpha
Subjects = 30
Raters = 2
alpha = 0.876
>
> # Overall, "ordinal" performs better than "nominal", DW and YZ have a higher alpha.
>
> ```
Error: attempt to use zero-length variable name
Execution halted