Evaluating Online Data Collection Platforms Using A Simple Rule-Following Task
通过一个简单的规则遵循任务（硬币任务），比较了Amazon Mechanical Turk、Prolific Academic和德国大学实验室的数据质量，发现MTurk的合规率显著低于其他两个平台。
High-quality experimental data is crucial for human-subject research. The increasing prevalence of online experimental studies has raised concerns regarding the quality and reliability of data collected in such environments. Recent evidence indicates that data quality varies across popular crowdsourcing platforms. We test if this also holds for less complex, easily comprehensible tasks. We find that compliance rates in a simple rule-following task are significantly lower in our Amazon Mechanical Turk sample compared to those from Prolific Academic and a German university lab. • Behavioral data quality varies across online crowdsourcing platforms. • Participants conduct a simple rule-following task, namely the coins-task. • We recruit participants on MTurk, Prolific and from a university lab database. • Data quality and rule compliance on MTurk is different compared to the other two. • Our findings align with other studies showing decreased data quality on MTurk.