Testing and interpreting replication studies: Insights from confidence intervals and Bayesian analyses
提出多阶段置信区间检验和贝叶斯分析两种方法,克服了传统并排比较的局限,帮助研究者更全面地评估复制研究结果与原研究的差异。
Testing and interpreting replications typically focus on whether they reproduce the sign (direction) and the statistical significance of the parameter estimates reported in a focal study. These side-by-side comparisons of replication and original study findings often yield a dichotomous “successful” or “unsuccessful” replication outcome. However, such practices can lead to incorrect conclusions, lose information by failing to reveal how and to what degree replication findings differ from those of the original research, and threaten replication’s role in safeguarding and advancing empirical literatures. This paper presents two approaches—a multi-staged set of confidence interval tests and Bayesian analyses—which use findings from original and replication studies as inputs for additional assessments of replicability. These methods overcome the limitations of side-by-side comparisons and provide more complete insights into the meaning and boundaries of the original study’s findings. Guidelines for their use are provided, and an empirical example is reported to provide concrete step-by-step illustrations.