Accelerated Gradient Descent by Concatenation of Stepsize Schedules
本文提出一种拼接步长计划的新方法，构造两种新的步长调度，在光滑凸优化上达到O(n^{-\log_2(\sqrt{2}+1)})收敛率，其常数优于现有数值结果，适合关注梯度下降收敛速度的研究者。
This work considers stepsize schedules for gradient descent on smooth convex objectives. We extend the existing literature and propose a unified technique for constructing stepsizes with analytic bounds for an arbitrary number of iterations. This technique constructs new stepsize schedules by concatenating two stepsize schedules with fewer steps. Using this approach, we introduce two new families of stepsize schedules, achieving a convergence rate of $O(n^{-\log_2(\sqrt 2+1)})$ with state-of-the-art constants for the objective value and gradient norm of the last iterate, respectively. Furthermore, our analytically derived stepsize schedules either match or surpass the existing best numerically computed stepsize schedules.