The structure of surveys and the peril of panels
研究了大学排名中维度问题和抽样偏差的交互作用,以《泰晤士高等教育学生体验调查》为例,指出面板设计可能导致对学生体验质量的扭曲评估,提醒政策制定者注意样本不具代表性的风险。
University league tables give the image that there is a single dimension along which institutions can be placed. Most derive rankings from an aggregate score of multiple items, which often include opinion responses which has the potential to introduce sampling bias. This paper explores what happens when the two issues of dimensionality and sampling bias interact in league tables. It uses the Times Higher Education Student Experience Survey – which uses a panel design – to explore these issues. It notes that they combine to produce a distorted image of the relative quality of the student experience in different institutions. We conclude that ignoring dimensionality and the systematically unrepresentative nature of the sample could lead policy makers to draw inappropriate conclusions.