Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models
研究了认知科学中单隐层前馈网络模型的反向传播递归估计方法，给出了其几乎必然收敛和渐近正态的条件，并提出了一个统计效率更高的两步估计法。
Abstract We investigate the properties of a recursive estimation procedure (the method of "back-propagation") for a class of nonlinear regression models (single hidden-layer feedforward network models) recently developed by cognitive scientists. The results follow from more general results for a class of recursive m estimators, obtained using theorems of Ljung (1977) and Walk (1977) for the method of stochastic approximation. Conditions are given ensuring that the back-propagation estimator converges almost surely to a parameter value that locally minimizes expected squared error loss (provided the estimator does not diverge) and that the back-propagation estimator is asymptotically normal when centered at this minimizer. This estimator is shown to be statistically inefficient, and a two-step procedure that has efficiency equivalent to that of nonlinear least squares is proposed. Practical issues are illustrated by a numerical example involving approximation of the Hénon map. Key Words: Back-propagationDeterministic chaosFlexible functional forms m EstimationMisspecified modelsNeural networksNonlinear regressionRecursive estimation