Interrater Agreement Reconsidered: An Alternative to the rwg Indices
针对连续变量评分者间一致性最常用的rwg(1)指数存在缺陷，本文提出新统计量awg(1)，它类似于Cohen's kappa，并比较了不同方法，为研究者选择合适的一致性指标提供建议。
For continuous constructs, the most frequently used index of interrater agreement (r wg(1))can be problematic. Typically, rwg(1) is estimated with the assumption that a uniform distribution represents no agreement. The authors review the limitations of this uniform nullr wg(1) index and discuss alternative methods for measuring interrater agreement. A new interrater agreement statistic,a wg(1), is proposed. The authors derive thea wg(1) statistic and demonstrate thatawg(1) is an analogue to Cohen’s kappa, an interrater agreement index for nominal data. A comparison is made between agreement estimates based on the uniformr wg(1) and a wg(1), and issues such as minimum sample size and practical significance levels are discussed. The authors close with recommendations regarding the use ofr wg(1) /rwg(J) when a uniform null is assumed,r wg(1) /rwg(J) indices that do not assume a uniform null,awg(1) / a wg(J) indices, and generalizability estimates of interrater agreement.