通过嗅探测试审视发表过程中的选择压力

Examining Selection Pressures in the Publication Process through the Lens of Sniff Tests

Review of Economics and Statistics · 2024
被引 0
人大 AFT50ABS 4

中文导读

研究分析了经济学论文中平衡检验、预趋势检验等“嗅探测试”的发表过程,发现超过30%的随机对照试验平衡检验被移除,其他测试的误设率超过40%。

Abstract

Abstract Economics papers increasingly report balance, pre-trend, placebo, and other “sniff tests,” rejection of which is bad news for authors, undermining the credibility of their main results. We derive nonparametric bounds on the latent proportion of significant sniff tests removed by the publication process (whether by p-hacking or relegation to the file drawer) and the proportion whose significance was due to true misspecification, not bad luck. Using a hand-collected sample of nearly 30,000 sniff tests, we estimate a removal rate of over 30% for balance tests in randomized controlled trials and a misspecification rate of over 40% for other tests.

嗅探检验发表压力发表偏倚误设率