Assessing Interrater Agreement via the Average Deviation Index Given a Variety of Theoretical and Methodological Problems
针对组织与应用心理学研究中常见的理论与方法论问题，给出了平均偏差指数的临界值和零范围，帮助研究者判断评分者间一致性的实际意义和理论一致性。
Currently, guidelines do not exist for applying interrater agreement indices to the vast majority of methodological and theoretical problems that organizational and applied psychology researchers encounter. For a variety of methodological problems, we present critical values for interpreting the practical significance of observed average deviation (AD) values relative to either single items or scales. For a variety of theoretical problems, we present null ranges for AD values, relative to either single items or scales, to be used for determining whether an observed distribution of responses within a group is consistent with a theoretically specified distribution of responses. Our discussion focuses on important ways to extend the usage of interrater agreement indices beyond problems relating to the aggregation of individual level data.