Don’t Forget the Items
提出用条目层面元分析和实质效度分析来重新评估量表的构念效度,并以任务绩效和组织公民行为量表为例,发现部分条目表现不如初始验证结果。
Researchers are generally advised to provide rigorous item-level construct validity evidence when they develop and introduce a new scale. However, these precise, item-level construct validation efforts are rarely reexamined as the scale is put into use by a wider audience. In the present study, we demonstrate how (a) item-level meta-analysis and (b) substantive validity analysis can be used to comprehensively evaluate construct validity evidence for the items comprising scales. This methodology enables a reexamination of whether critical item-level issues that may have been supported in the initial (often single study) scale validation process—item factor loadings and theorized measurement model fit, as examples—hold up in a larger set of heterogeneous samples. Our demonstration focuses on a commonly used scale of task performance and organizational citizenship behavior, and our findings reveal that several of the items do not perform as may have been suggested in the initial validation effort. In all, our study highlights the need for researchers to incorporate item-level assessments into evaluations of whether construct scales perform as originally promised.