IRT-Based Assessments of Rater Effects in Multiple-Source Feedback Instruments
比较三种基于项目反应理论的模型在360度反馈中评估测量等价性的效果，发现传统DIF方法能提供最多关于评分者能力概念的信息，而其他模型可估计评分者宽严度，但评分者来源效应虽显著却对得分影响不大。
This study compares three item response theory–based models of assessing measurement equivalence in 360° feedback: the traditional differential item functioning (DIF) methodology, Muraki’s rater’s effect model, and Patz, Junker, and Johnson’s hierarchical rater model. Using data from 491 managers collected on the Benchmarks instrument, the authors found that the traditional DIF methodology provides the most information about the rater’s conception of the ratee’s ability, whereas the other two models provide explicit estimates of rater leniency/severity. The authors also found that rater source effects of leniency and severity, even though statistically significant, did not substantially affect the observed score at the item and scale levels. The different results and conclusions produced by each model are discussed.