Heteroscedasticity-aware stratified sampling to improve uplift modeling
提出一种异方差感知的分层抽样方法，通过从高噪声个体中多采样来提高平均处理效应估计精度和提升模型性能，适用于商业实验数据有限的情形。
Randomized controlled trials (RCTs) are conducted in many business applications including online marketing or customer churn prevention to investigate the effect of specific treatments (coupons, retention offers, mailings, etc.). RCTs allow for the estimation of average treatment effects and the training of (uplift) models for the heterogeneity of treatment effects across individuals. The problem with RCTs is that they are costly, and this cost increases with the number of individuals included. These costs have inspired research on how to conduct experiments with a small number of individuals while still obtaining precise treatment effect estimates. We contribute to this literature a heteroskedasticity-aware stratified sampling (HS) scheme. We leverage the fact that different individuals have different noise levels in their outcome and that precise treatment effect estimation requires more observations from the ”high-noise” individuals than from the ”low-noise” individuals. We show theoretically and empirically that HS sampling yields significantly more precise estimates of the ATE, improves uplift models, and makes their evaluation more reliable compared to RCT data sampled completely randomly. Due to these benefits and the simplicity of our approach, we expect HS sampling to be valuable in many real-world applications in business and beyond. • In many uplift modeling applications, randomized experiments are conducted to obtain data. • Such data sets often have binary outcomes and a low proportion of positive outcomes. • As a consequence, these data sets often exhibit strong heteroskedasticity. • Our sampling procedure leverages heteroskedasticity for efficient experiment sample selection. • The procedure simultaneously improves uplift model training and evaluation.