Some Unified Theory for Variance Reduced Prox-Linear Methods
这项研究针对非凸非光滑复合优化问题，内层映射可作方差缩减。它提出统一的收敛理论，覆盖多种方差缩减向量和雅可比构造，要求条件弱，并给出高概率保证，适合研究优化算法的学者参考。
Abstract. This work considers the nonconvex, nonsmooth problem of minimizing a composite objective of the form [Formula: see text] where the inner mapping [Formula: see text] is a smooth finite summation or expectation amenable to variance reduction. In such settings, prox-linear methods enjoy variance-reduced speed-ups despite the existence of nonsmoothness. We provide a unified convergence theory applicable to a wide range of common variance-reduced vector and Jacobian constructions. All the technical conditions we require for variance-reduced methods can be summarized in a single unified assumption. Our theory (i) requires only operator norm bounds on Jacobians (whereas prior works used potentially much larger Frobenius norms), (ii) provides state-of-the-art high probability guarantees, and (iii) allows inexactness in proximal computations.