Convergence Analysis of Regularized Elman Neural Networks Under Relaxed Conditions
本文提出一种带正则化项的Elman神经网络模型，证明其反向传播算法在宽松条件下的收敛性，并放宽了对权重有界性、学习率和误差函数稳定集的要求，仿真验证了算法性能。
Recurrent neural networks (RNNs) have been found to be a promising field of research for time series prediction, continuous-time system modeling, and discrete-time sequence data processing. The Elman network can be viewed as an RNN with a single layer of feedback connections and local storage units. However, the Elman network has two limitations that restrict its application. One is the overfitting phenomenon, which limits its generalization ability. The other is the complexity of the network structure, which makes its theoretical results incomplete and limits its practicality. The main contribution of this article is to propose a new Elman network model with a regularization term. First, we demonstrate the convergence of the backpropagation algorithm with weight decay regularization under some reasonable conditions. Second, contrary to the usual requirement for bounded weights, we show that such boundedness is no longer a necessary condition for convergence analysis. Third, we show that the requirements for learning rate and the stable set of error functions can also be relaxed. Finally, we provide simulation results to demonstrate the excellent performance of our proposed algorithm.