Enhanced Balancing of Bias-Variance Tradeoff in Stochastic Estimation: A Minimax Perspective
提出一种新框架，通过定义渐近极小极大风险比来构造随机估计量，使其在未知模型参数下始终优于现有基准，并给出一种显式加权方案。
In “Enhanced Balancing of Bias-Variance Tradeoff in Stochastic Estimation: A Minimax Perspective”, the authors study a framework to construct new classes of stochastic estimators that can consistently beat existing benchmarks regardless of key model parameter values. Oftentimes biased estimators, such as finite-difference estimators in black box stochastic gradient estimation, require selection of tuning parameters to balance bias and variance and ultimately minimize overall errors. Unfortunately, this relies on model knowledge that is unknown a priori and thus leads to ad hoc choices in practice. The authors introduce a new notion called asymptotic minimax risk ratio, which is designed to compare new estimators against existing benchmarks, whose values less than one imply that the new estimators could asymptotically outperform the benchmarks regardless of the model parameter value. Based on this, the authors study an outperforming weighting scheme by explicitly analyzing the asymptotic minimax risk ratio via a tractable reformulation of a nonconvex optimization problem.