Novel Discounted Adaptive Critic Control Designs With Accelerated Learning Formulation
受逐次松弛法启发，提出一种新型折扣迭代自适应动态规划框架，使值函数序列具有可调收敛速度，并设计加速学习算法，在球杆平衡系统上验证了其相比传统值迭代能大幅加速收敛并降低计算成本。
Inspired by the successive relaxation method, a novel discounted iterative adaptive dynamic programming framework is developed, in which the iterative value function sequence possesses an adjustable convergence rate. The different convergence properties of the value function sequence and the stability of the closed-loop systems under the new discounted value iteration (VI) are investigated. Based on the properties of the given VI scheme, an accelerated learning algorithm with convergence guarantee is presented. Moreover, the implementations of the new VI scheme and its accelerated learning design are elaborated, which involve value function approximation and policy improvement. A nonlinear fourth-order ball-and-beam balancing plant is used to verify the performance of the developed approaches. Compared with the traditional VI, the present discounted iterative adaptive critic designs greatly accelerate the convergence rate of the value function and reduce the computational cost simultaneously.