Adjustable Iterative Q-Learning Schemes for Model-Free Optimal Tracking Control
提出一种收敛速度可调的确定性值迭代Q学习算法，用于完全未知非仿射系统的轨迹跟踪，通过离线数据更新参数，减少迭代次数和计算负担。
This article puts emphasis on the deterministic value-iteration-based <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -learning (VIQL) algorithm with adjustable convergence speed, followed by the application verification on trajectory tracking for completely unknown nonaffine systems. It is worth emphasizing that, under the effect of learning rates, the convergence speed can be adjusted and the new convergence criterion of the VIQL framework is investigated. The merit of the adjustable VIQL scheme is that it can quicken the learning speed and decrease the number of iterations, thereby reducing the computation burden. To carry out the model-free VIQL algorithm, the offline data of system states and reference trajectories are collected to provide the reference control, the tracking error, and the tracking control, which promotes the parameter updating of the adjustable VIQL algorithm via the off-policy learning scheme. By this updating operation, the convergent optimal tracking policy can guarantee that arbitrary initial state tracks the desired trajectory and can completely obviate the terminal tracking error. Finally, numerical simulations are conducted to indicate the validity of the designed tracking control algorithm.