Designing grant-review panels for better funding decisions: Lessons from an empirically calibrated simulation model
通过经验校准的计算机模拟模型，研究评审小组的评分和决策因素如何影响提案选择的正确性，并提出改进小组设计的干预措施。
To explore how factors relating to grades and grading affect the correctness of choices that grant-review panels make among submitted proposals. To identify interventions in panel design that may be expected to increase the correctness of choices. Experimentation with an empirically-calibrated computer simulation model of panel review. Model parameters are set in accordance with procedures at a national science funding agency. Correctness of choices among research proposals is operationalized as agreement with the choices of an elite panel. The simulation model generates several hypotheses to guide further research. Increasing the number of grades used by panel members increases the correctness of simulated choices among submitted proposals. Collective decision procedures giving panels a greater capacity for discriminating among proposals also increase correctness. Surprisingly, differences in grading standards among panel members do not appreciably decrease correctness.