Unpacking p-Hacking and Publication Bias
利用期刊投稿的独特数据,识别并解析发表偏倚和p值操纵,发现初始投稿存在显著聚集,但同行评审过程对检验统计量分布影响甚微,发表偏倚的普遍性可能不如想象中严重。
We use unique data from journal submissions to identify and unpack publication bias and p-hacking. We find initial submissions display significant bunching, suggesting the distribution among published statistics cannot be fully attributed to a publication bias in peer review. Desk-rejected manuscripts display greater heaping than those sent for review; i.e., marginally significant results are more likely to be desk rejected. Reviewer recommendations, in contrast, are positively associated with statistical significance. Overall, the peer review process has little effect on the distribution of test statistics. Lastly, we track rejected papers and present evidence that the prevalence of publication biases is perhaps not as prominent as feared.