Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training
提出一类无需计算目标函数的多层无约束非线性优化算法，可降低对噪声的敏感性和计算成本，并在深度神经网络监督学习训练中验证其效果。
A class of multilevel algorithms for unconstrained nonlinear optimization is presented which does not require the evaluation of the objective function. The class contains the momentum-less AdaGrad method as a particular (single-level) instance. The choice of avoiding the evaluation of the objective function is intended to make the algorithms of the class less sensitive to noise, while the multilevel feature aims at reducing their computational cost. The evaluation complexity of these algorithms is analyzed and their behavior in the presence of noise is then illustrated in the context of training deep neural networks for supervised learning applications.