Implicit criteria weighting and evaluators' backgrounds in grant peer review
研究分析了2105份评审报告，发现评审者给研究质量赋予的权重约为社会相关性的两倍，且这种权重模式在不同背景评审者中稳定存在，PI资历影响较小，仅在评审者信心不足时被用作替代标准。
Grant peer review (GPR) is a cornerstone of modern research funding, yet the process by which reviewers integrate multiple criteria into a single overall evaluation—known as commensuration —remains poorly understood. We study the implicit weights that reviewers assign to PI qualifications, research quality, and societal relevance, using a unique dataset of 2105 review reports for 586 proposals submitted to the Swedish Foundation for Strategic Research (2011–2017), complemented by manual coding of comments and interviews with screeners and funding officers. Commensuration influences roughly 29% of final scores. The research quality criterion dominates evaluations, receiving about twice the weight of relevance. This commensuration pattern is surprisingly stable across pools of reviewers from different fields and geographic areas and irrespective of the reviewers' profile and their presumed ability to assess relevance. PI qualifications generally play a minor role, tempering concerns about pervasive reputational advantages, and tend to be used more when evaluators feel less competent, hinting that reputation is used as a fallback consideration when the reviewer lacks confidence. Overall, the results suggest that commensuration may systematically contribute to prioritizing technical merit over relevance even beyond the intentions of the funders.