The Restriction of Variance Hypothesis and Interrater Reliability and Agreement: Are Ratings from Multiple Sources Really Dissimilar?
研究发现，360度评估中不同来源评分之间的低相关性可能源于统计伪像（绩效方差受限），而非真正提供独特信息；模拟和实地研究显示，不同来源间的一致性几乎与来源内部一样高。
The fundamental assumption underlying the use of 360-degree assessments is that ratings from different sources provide unique and meaningful information about the target manager’s performance. Extant research appears to support this assumption by demonstrating low correlations between rating sources. This article reexamines the support of this assumption, suggesting that past research has been distorted by a statistical artifact—restriction of variance in job performance. This artifact reduces the amount of between-target variance in ratings and attenuates traditional correlation-based estimates of rating similarity. Results obtained from a Monte Carlo simulation and two field studies support this restriction of variance hypothesis. Noncorrelation-based methods of assessing interrater agreement indicated that agreement between sources was about as high as agreement within sources. Thus, different sources did not appear to be furnishing substantially unique information. The authors conclude by questioning common practices in 360-degree assessments and offering suggestions for future research and application.