Error Bound Analysis of $Q$ -Function for Discounted Optimal Control Problems With Policy Iteration
分析了未知离散时间非线性系统折扣最优控制问题中，策略迭代算法下Q函数的误差界，证明了近似Q函数在给定有界条件下收敛到最优Q函数的有限邻域。
In this paper, we present error bound analysis of the Q-function for the action-dependent adaptive dynamic programming for solving discounted optimal control problems of unknown discrete-time nonlinear systems. The convergence of Q-functions derived by a policy iteration algorithm under ideal conditions is given. Considering the approximated errors of the Q-function and control policy in the policy evaluation step and policy improvement step, we establish error bounds of approximate Q-functions in each iteration. With the given boundedness conditions, the approximate Q-function will converge to a finite neighborhood of the optimal Q-function. To implement the presented algorithm, two three-layer neural networks are employed to approximate the Q-function and the control policy, respectively. Finally, a simulation example is utilized to verify the validity of the presented algorithm.