Institutional design and biases in evaluation reports by international organizations
研究了联合国系统9个国际组织的1082份评估报告,发现由业务部门委托的报告比中央评估部门的报告包含更多正面评价,说明制度设计会影响评估结果的客观性。
Abstract Governments spend hundreds of millions on evaluations to assess the performance of public organizations. In this article, we scrutinize whether variation in the institutional design of evaluation systems leads to biases in evaluation findings. Biases may emerge because influence over evaluation processes could enable the bureaucracy to present its work in a more positive way. We study evaluation reports published by nine international organizations (IOs) of the United Nations system. We use deep learning to measure the share of positive assessments at the sentence level per evaluation report as a proxy for the positivity of evaluation results. Analyzing 1082 evaluation reports, we find that reports commissioned by operative units, as compared to central evaluation units, systematically contain more positive assessments. Theoretically, this link between institutional design choices and evaluation outcomes may explain why policymakers perceive similar tools for evidence‐based policymaking as functional in some organizations, and politicized in others.