随机优化中的渐近最优性

Asymptotic optimality in stochastic optimization

Annals of Statistics · 2021
被引 27
ABS 4★

中文导读

研究了随机凸优化的局部复杂度度量,建立了类似经典统计中Hájek-Le Cam的局部极小极大理论,并开发了自适应达到最优收敛率的在线方法。

Abstract

We study local complexity measures for stochastic convex optimization problems, providing a local minimax theory analogous to that of Hájek and Le Cam for classical statistical problems. We give complementary optimality results, developing fully online methods that adaptively achieve optimal convergence guarantees. Our results provide function-specific lower bounds and convergence results that make precise a correspondence between statistical difficulty and the geometric notion of tilt-stability from optimization. As part of this development, we show how variants of Nesterov’s dual averaging—a stochastic gradient-based procedure—guarantee finite time identification of constraints in optimization problems, while stochastic gradient procedures fail. Additionally, we highlight a gap between problems with linear and nonlinear constraints: standard stochastic-gradient-based procedures are suboptimal even for the simplest nonlinear constraints, necessitating the development of asymptotically optimal Riemannian stochastic gradient methods.

随机优化凸优化统计学习理论随机梯度方法