Rater agreement, accuracy, and experienced cognitive load: Comparison of distributional and traditional assessment approaches to rating performance
通过两个实验室研究比较分布评估与传统评估方法，发现分布评估能提高评分者间一致性和准确性，但评分者报告了更高的认知负荷，且认知负荷未中介评分格式与准确性的关系。
Error in performance ratings is typically believed to be due to the cognitive complexity of the rating task. Distributional assessment (DA) is proposed to improve rater accuracy by reducing cognitive load. In two laboratory studies, raters reported perceptions of cognitive effort and difficulty while assessing rating targets using DA or the traditional assessment approach. Across both studies, DA raters showed greater interrater agreement, and Study 2 findings provide some support for DA being associated with greater true score rating accuracy. However, DA raters also reported experiencing greater cognitive load during the rating task, and cognitive load did not mediate the relationship between rating format and rater accuracy. These findings have important implications regarding our understanding of cognitive load in the rating process.